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,,1^ ■ Abstract: Wo consider the asymptotics of the invariant measure for the 

process of the empirical spatial distribution of A'^ coupled Markov chains 
in the limit of a large number of chains. Each chain reflects the stochastic 
evolution of one particle. The chains arc coupled through the dependence 
of the transition rates on this spatial distribution of particles in the various 
states. Our model is a caricature for medium access interactions in wireless 

Cn ' local area networks. It is also applicable to the study of spread of epi- 

^ ' demies in a network. The limiting process satisfies a deterministic ordinary 

^SJ , differential equation called the McKean-Vlasov equation. When this differ- 

^+ , ential equation has a unique globally asymptotically stable equilibrium, the 

spatial distribution asymptotically concentrates on this equilibrium. More 
generally, its limit points are supported on a subset of the cj-limit sets of the 
McKean-Vlasov equation. Using a control-theoretic approach, we examine 

C^~" the question of large deviations of the invariant measure from this limit. 
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1. Introduction 

Spurred by the seminal work of Bianchi [5] , there has been a flurry of aetivity 
in the communication networks community on mean field models for carrier 
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sense multiple access (CSMA) protocols and their large time behavior. The 
continuous-time model for the wireless local area network (WLAN) is as follows. 
There are N particles (nodes) in the network. At each instant of time, a particle's 
state is a particular value taken from the finite state space Z = {0, 1, ■ • • , r — 1}. 
A particle's state represents the number of failed attempts at transmission of the 
head-of-the-line packet at that particle's queue. When a particle is in state i, a 
successful transmission gets the packet out of the system, and the particle moves 
to state to service the next packet. A failed transmission moves the particle to 
state i + 1 (mod r). In the case when i was initially r — 1, i.e., r — 1 unsuccessful 
transmission attempts were already made, another failed attempt results in the 
discarding of the packet. The particle then moves to state with the next 
packet readied for transmission. We may interpret r as the maximum number of 
transmission attempts. The transition rate for a particle from state i to state j 
is governed by mean field dynamics, i.e., the transition rate is Xij{p,N{t)) where 

/xjv (t) is the empirical distribution of the states of particles at time t. If Xn (t) 
is the state of the nth particle at time t, then one may write ^Ar(t) as 

1 ^ 

n=l 

The particles interact only through the dependence of their transition rates on 
the current empirical measure /^Ar(i). 

The transitions allowed in the above model are i to cither t + 1 (mod r) or 
0. Let us say that £ denotes the set of allowed transitions. In the above model, 

£ = {{i,i + l),i = 0, 1,--- ,r-l}U{(i,0),i = 0, !,•■• ,r-l} 

where the addition is taken modulo r. 

The process X'^^\-) = {Xn '{■), 1 < n < N} is clearly a Markov process. 
But one difficulty needs to be surmounted in analyzing this system: the size of 
the state space grows exponentially in the number of particles. A step towards 
addressing this difficulty is to consider the evolution or flow of the empirical 
measure over time, which we shall call empirical process. This is a stochastic Li- 
ouville equation that lives on a smaller state space. In the infinite particle limit, 
this evolution turns out to be deterministic and is given by the McKean-Vlasov 
equation, whose large time behavior is an indicator of what one might expect of 
a finite but large population. In particular, if the deterministic evolution, given 
by the McKean-Vlasov equation, has a unique globally asymptotically stable 
equilibrium, then the states of a finite number of tagged particles are asymptot- 
ically independent and their joint law is given by the product of this equilibrium 
measure. The idea in fact was introduced by Kac as a simple model in kinetic 
theory [23] and was later studied by McKean and others (see, e.g., [25]). See 
[32] for an extensive account and [22] for a treatment of processes with jumps. 

Several papers have provided rigorous analyses of Bianchi's heuristic for 
studying WLANs. See, e.g., [9], [27], [8], [33], [3], [26]. See [16] for an excel- 
lent survey, [31], [1], [2] for early precursors, and [4] for an apphcation of the 
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same technique in game theory. As remarked in [16], experimental evidence for 
CSMA protocols indicates that the model and the predictions made by the anal- 
yses are surprisingly accurate even for small populations. Indeed, this is one of 
the main reasons for the model's enormous popularity. As a step in the direction 
of understanding this concentration phenomenon, we study the continuous-time 
model in this paper with the following goals. 

1. Obtain a large deviation principle over finite time durations for the se- 
quence of empirical measures and empirical flow, uniformly in the initial 
condition. This is of course rough asymptotics for large N, but suggests 
exponentially fast convergence to the deterministic limit. 

2. Obtain a large deviation principle for the sequence of invariant measures, 
with single or multiple stable limit sets. This helps resolve which of the 
several equilibria, if present, will be selected in the large N limit. It also 
helps understand metastable behavior in such systems. 

3. Provide a control theoretic framework to solve the problem of invariant 
measure in order to expose its strength and limitation. As we will see, we 
can go quite some distance using this deterministic approach, but eventu- 
ally need to study the noisy system for resolution of some degeneracies. 

It must be noted that the above continuous-time model does not perfectly 
capture all aspects in a WLAN. In particular, interactions and changes of states 
occur in discrete-time units of slots in WLANs, and multiple nodes may transit 
in one slot. Multiple transitions never occur, almost surely, in our continuous- 
time model. Nevertheless, if the discrete-time model's transition rates and the 
slot sizes are appropriately scaled down as A^ grows so that the transition rates 
approach constants, our continuous time model provides accurate predictions 
of behavior on the discrete-time model. Our model also has wider applicability, 
one example being the study of spread of epidemics in a network; see [15, Sec. 
2.4]. 

Large deviation principles over compact time durations for interacting diffu- 
sions and interacting jump Markov processes have been well-studied by several 
authors, e.g., [11], [24], [15], [19], [10], [14]. The works [15], [10], [14] establish 
large deviation principles for empirical measures in path space over finite time 
durations and characterize the rate functions, while [19] considers infinite time 
durations under the inductive topology. The works [11], [24], [15], [10], and [20] 
also study large deviation of the empirical process from the McKean-Vlasov 
limit, over finite time durations or infinite time durations under the inductive 
topology ([20]). The rate function measures the difficulty of passage of the em- 
pirical process in the neighborhood of a deviating path. 

For a fixed N, when t — )■ +cxd, the stationary or invariant measure is of 
interest. When the limiting McKean-Vlasov dynamics has a unique globally 
asymptotically stable equilibrium, the invariant measure converges weakly in 
the infinite particle limit to the point mass at the equilibrium of the McKean- 
Vlasov dynamics (see, e.g., [3]). When there are multiple equilibria, the invariant 
measure concentrates on a subset of the w-limit sets for the McKean-Vlasov 
dynamics ([21, Ch. 6] and [4]). Large deviations from this limit have been well- 
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studied (sec [21, Ch. 6], [30]). As indicated earlier, large deviation results for 
the invariant measure help resolve which of several possible w-limit sets may 
be selected in the limit of a large number of nodes. They also help understand 
and predict metastable behavior in such systems. If the system is trapped in 
an undesirable equilibrium, exit times from domains and likely paths can be 
predicted. 

Our approach for solving the large deviations of invariant measure exploits 
a control-theoretic view described in [6], which considers a class of diffusions 
and generalizes results of [29] and [12] on small noise asymptotics of invariant 
measures and exit probabilities in diffusions. See [28] for small noise asymptotics 
of exit probabilities in processes with jumps. The control-theoretic approach 
differs from those of Freidlin and WentzcU [21] and Shwartz and Weiss [30]. The 
latter rely on a study of an embedded Markov chain of states at hitting times 
of neighborhood of the stable limit sets (see, e.g., [21, Ch. 6.4]). Our control- 
theoretic approach is, in a sense, a noise-free approach. It gives a complete 
solution when there is a unique globally asymptotically stable equilibrium for the 
McKean-Vlasov dynamics. When dealing with multiple w-limit sets though, our 
approach requires the study of the noisy system associated with the embedded 
Markov chain described above, for resolution of certain boundary conditions. 

We now outline our main arguments and describe the paper's organization. 

We begin with a formal description of the model in Section 2.1. In Section 
2.2, we first establish a large deviation principle for empirical measures of paths 
over finite durations. For pure jump processes with interactions, this was estab- 
hshed by [24], [15], [19], [10], [14], as indicated earlier. While [24] considers a 
fixed initial condition for all the particles, [10], [19], and [14] consider random, 
independent, and identically distributed (i.e., chaotic) initial conditions for the 
particles. To establish the large deviation principle for the interacting system, 
they first establish the large deviation principle for the independent and nonin- 
teracting system via Sanov's theorem, then exploit the Girsanov transformation 
to describe the probability measure for the interacting case, and then apply the 
Laplace- Varadhan principle. In order to eventually pass to the invariant mea- 
sure, we need to establish a stronger uniform large deviation principle when the 
initial conditions of the particles are such that the initial empirical measures 
converge weakly, but are otherwise arbitrary. (See the remark following [19, Th. 
4.1].) The limiting empirical measure defines the initial condition for the limiting 
deterministic McKean-Vlasov dynamics. This stronger uniform large deviation 
principle alluded to above is available for diffusions with mean field interactions 
in [11] and for certain classes of jump processes in [15] where the holding times 
alone are mediated by the interaction and not the jump probabilities. Our result 
is a mild extension facilitated by a generalization of Sanov's theorem given in 
[11]. While this part of our result is not surprising, we could not find a ready 
reference in the literature, and so we state the result in Section 2.2 and provide 
a proof in Section 3 based on the approach in [24]. We reemphasize that this 
result is only for finite durations, and is only a step towards addressing our next 
goal of asymptotics of the invariant measure. 

In Section 2.3, we apply the contraction principle to obtain results on a large 
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deviation from the McKean-Vlasov limit. As in [24], wo prove this under the 
finer uniform norm topology. In Section 2.4, we once again apply the contraction 
principle to argue a large deviation principle for terminal measure. We then 
establish some crucial estimates on the rate function for use in later sections. 
Finally in this section, we argue that if the initial measures satisfy a large 
deviation principle, then so do the joint initial and terminal measures. 

In Sections 2.5 - 2.7, we study the large deviation principle for the invariant 
measure. To do this, we first establish a subsequential large deviation principle, 
then argue that the rate function satisfies an integral version of the Hamilton- 
Jacobi-Bellman equation for a particular control problem, and then finally show 
that the rate function is unique if the associated McKean-Vlasov dynamics has 
a unique globally asymptotically stable equilibrium. The arguments to establish 
the uniqueness of the rate function parallel those of [6] with the significant 
difference that while the Hamilton- Jacobi-Bellman equation was arrived at in [6] 
via the theory of viscosity solutions, here we use the contraction principle. While 
this provides a complete result in case of a single globally asymptotically stable 
equilibrium, it leaves some indeterminacy regarding uniqueness of the so called 
'potential function' whose global minima the invariant measure concentrates on 
in the large N limit. To resolve this, one has to fall back upon the framework 
of Frcidlin and Wentzell [21, Ch.6] with minor modifications. The latter are 
detailed in the Appendix. 

Proofs that are not central to our control-theoretic view point are relegated 
to Sections 3, 4, 5, and 6. The Appendix details the minor modifications to the 
arguments of Freidlin and Wentzell [21, Ch.6] for resolution of the rate function 
values at the stable limit points. 

2. The model and main results 

Consider N interacting Markov chains denoted by 

xW(t), l<n<A^, i>0, 

on a finite state space Z := {0, • • • , r — 1}, with dynamics as follows. Let fJ-Nit) 
be the empirical measure of the particles at time t, i.e.. 



N 
J- ■ 

MAT 



1 ^ 



N 

n=l 

Here and later 7V(i(- • • ) will denote the set of probability measures on "• • • " 
with an associated a-algebra that will be clear from the context. Let f be a set 
of directed edges (i, j) G Z x Z defining an irreducible directed graph G{Z,£). 
The subset Zi = {j G Z such that {i,j) S £} is the neighbor set of i. A particle 
n in state i transits to state j G Zi with a rate Xij{^Nit)) that depends on the 
states of the other particles only through the empirical measure at that time. 
Thus the processes interact only through the dependence of their transition 
rates on the current empirical measure. Further assumptions will soon be made 
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on the rates Xi,j{-) for {i,j) £ £■ Our study is on the large N asymptotics of 
the process /XAr(-) and the asymptotics of its invariant measure. 

The first step in our approach to this asymptotics is a large deviation principle 
in path space, obtained from Sanov's theorem. Towards this end, we follow [24] 
and formally define the model. 

2.1. Formal description 

Let T > 0, and let Xn : [0, T] — > Z denote the process of evolution of particle 
n over time. This is an element of the set D{[0,T],Z) of all cadlag paths from 
[0,T] to Z equipped with the Skorohod topology. We set the path to be left 
continuous at T. Let 

x^ = (xW,i < n < iV) e D([o,r],z^) 

denote the full description of paths of all N particles. The initial condition at 
time t = is 2^ = (z„, 1 < 7i < N). 

For notational convenience, we treat Z as the group Z^ with modulo r addi- 
tion, and define A^j- = j — i (mod r), so that j ~ i + Aij (mod r). Of course, a 
Aij transition can occur for a particle in state i to state j = i + Aij (mod r) 
only if j € Zi. Henceforth, we shall omit "mod r". 

The law P „ of X^ is a solution to the martingale problem on the space 
D{[0,T],Z'^) associated with the infinitesimal generator defined by 



A. 




($(a-^ + e„A,„,,)-$(x^)) 



N 

aw$(.^) = e e 

71=1 jez^^ 

(2.1) 
with domain Bi,{Z^) := the set of all bounded measurable functions on Z^ , 
and the initial condition z^ . In (2.1), x^ = (x„,l < n < N), e„ is the unit 
vector having 1 in the nth coordinate and Os elsewhere, and $ G Bh{Z^). The 
definition in (2.1) is a compact way of saying that particle n when in state 
Xn = i transits to state j = x„ + Ax^.j with a rate Xij{^N) that depends on 

the empirical measure ^n = i^/N) X]n'=i ^^n'- 
We make the following assumption: 

(A) There exist positive constants c > and C < +oo such that, for every 
(i,j) G £, the bounds c < Xij{-) < C hold, and moreover, the mapping 
M ^^ Aij(ju) is Lipschitz continuous over Aii{Z). 

The upper boundedncss of the rates implies that the martingale problem as- 
sociated with the generator A^'^\ domain Bb{Z^), and the initial condition 
z^ e Z^ admits a unique solution P^^^ e MiiD{[0,T],Z'^)) (sec for e.g., [17, 
Problem 4.11.15, p. 263]). 

We close this subsection with a remark on why our work on the invariant 
measure is not subsumed by the works of [30] and [21]. The process /iAr(-) 
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is obviously a Markov process with jumps. When a transition occurs, say a 
particle jumps from state i to state j, then //Ar(i)(i) decreases and fJ.N{t){j) 
increases by 1/7V. The transition rate for such a jump when the state is fJ-Nit) 
is (/iAr(i)(i))Ai.j(/iAr(i))- While \ij{^pf{t)) is indeed bounded away from zero, 
(/ZAr(i)(«))Aij(/xjv(t)) is not, and so the logarithm of these rates is not bounded, 
a requirement in [30]. Next, the process fiN takes values only in Mi{Z), and so 
when at the boundary Aii{Z), it is constrained to move only in those directions 
that keep it within A^i(Z). As a consequence, a finiteness condition [21, p. 146, 
II] on the associated Lagrangian function docs not hold. 

2.2. Empirical measure 

Let Gjv denote the mapping that takes the full description x^ to the empirical 
measure 



Gn ■■ {xn, l<n<N)e i?([0,r],Z^) ^l-J2S.,„e MiiD{[0,T],Z)). 

n=l 

Given the random variable X , the random empirical measure, denoted M^, 
is thus 

Mn = Gn{X^) e MiiDi[0,T],Z)). 

Clearly, the law of Mn depends on the initial condition z^ only through its 
empirical measure i^n = (1/-^) Sn=i '^z„- Write Pi^jv for the law of Mn, the 



push forward of P^n under the mapping Gn, i.e., Pi/„ = P[n ° Gj^^. 



The Ai 1 (Z)-valucd cadlag empirical process is 

N 



fiN-te [o,T] ^ fiNit) = ^ ^ \xir\t)} € -^l(^)' 

n=l 

and the corresponding mapping is denoted 

IN ■■ (xn, l<n<N)e D{[0,T],Z'') ^ (In : [0,T] ^ Mi{Z). 

Observe that /XAr(O) = vn, and that /iAr(i) is the projection ■nt{MN) at time t, 
and we write jjln = 7r(-/Vfjv). We thus have 

^1N = ^{Mn) = 7r(G^(X^)) = iNiX""). 

Consider now a hypothetical tagged particle. When there is no interaction, 
when all transition rates for {i,j) G £ transitions are unity, and when all other 
transition rates are 0, we can define the evolution of the tagged particle by the 
law Pz which is the unique solution to the martingale problem in D{[Q,T], Z) 
associated with the generator 

A°$(*) = ^ 1 • ($(j) - $(*)) 
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with domain Bi,{Z) and the initial condition z. The existence of a solution and 
the solution's uniqueness hold because the transition rates are upper bounded 
(see [17, Problem 4.11.15]). For any fixed n S D{[0,T],MiiZ)), let P^iti) be 
the unique solution to the martingale problem in D([0,T],Z) associated with 
the (time-varying) generator 

\(fM^) = E >^^AKt)) ■ mj) - $(0) (2.2) 

defined on Bi,{Z) and the initial condition z. Again, by the upper boundedness 
assumption on Ajj(-), Pzip-) is unique, and the density of Pzi^-) with respect to 
Pz can be written as (see [24, cqn. (2.4)]) 

^^(x)=exp{/ii(x;/i)} (2.3) 

dPz 

where 

hiix;fi)^ Y^ l{^^^^^_}logA:rt-,^t(Ai(^~-)) (2.4) 

0<t<T 






Consider the product distribution P°„ ' = ®n=\ -^z„ where the N particles 
evolve independently, with the nth particle's initial condition being z„. Again 

with i/N = (1/A^) J2n=i '^2"' ^^* -f^jv = IP°Jv ° ^N^- A simple application of 
Girsanov's formula yields that (see [24, eqn. (2.8)]) 

dP^^'> 

'^" (g) - cxp{iV/i(0)} (2.5) 



dP 



UN 

where h is related to /ii(-; ■) as follows: for a Q 6 Mi{D{[Q,T],Z)) 



HQ)^ / hi{x;7r{Q)) Q{dx). (2.6) 

JD{[0,T],Z) 

Let us define the spaces and topologies of interest. As in [24], consider the 
Polish space {X, d) where 

X = {xeD{[0,T],Z)\ E l{.(t)#..(t^} < +0O, 

0<t<T 

and for each t e (0,r] with x{t) ^ x{t—), we have x{t) S Zj,(j_)} 

with metric 

d{x,y) ^dswo{x,y) + \(p{x) - (p{y)\, x,y e X 
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where dsko stands for the Skorohod (complete) metric, and 

0<t<T 

denotes the number of jumps, ip is nonnegative and continuous (see [24. p. 299] 
For a function f : X ^ M., define 



ip = sup 



^ 1/(^)1 ^ 

denote 

C^{X) = {/ I / ; A' ^ M is continuous and ||/||,^ < +00} 

and 

Mi,^{X) = jo e Mi{X) I f ipdQ< +oo| . 

This is a subset of the algebraic dual Cip{X)* of C^p{X). Endow the set Ml.^p{X) 
with the weak* topology (t{Mi^^{X),C^{X)), the weakest topology under which 
Qn -^ Q as N ^ +CXD if and only if 

/ / dQM -^ I f dQ for each / e a^{X). 
Jx J X 

This topology is obviously finer than the topology of weak convergence in 
Mi{X). 

For a measure v £ Mi{Z), let P be the mixture given by 

dP{x) = Yl v{z)dP^{x). (2.7) 

and let P{ijl) be the mixture given by 

dP(M)(x) = ^K^)dP,(/x)(a;). (2.8) 

zeZ 

Define the relative entropy H : Ml.^p{X) — > [0, +00] of Q (with respect to P) 
as 

i?(Q|P) = P-^°^(-)'^ ^'^«.^ (2.9) 

I +00 otherwise. 

Also define the function J : AAi^ip{X) -^ [0, +00] 



J(Q) = sup 

f'^C:^(X) 



[ fdQ-Y, u{z) log / ef dP, 



(2.10) 



We first state a simple extension of [24, Th. 2.1]. See remarks following the 
statement on the nature of the extension. 
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Theorem 2.1. Suppose that the initial conditions vjq -^ v weakly. The sequence 
{Pu,^ ,N > 1) satisfies the large deviation principle in the space Mi.^p{X), 
endowed with the weak* topology (T{Aii^ip{X),C^{X)), with speed N and good 
rate function I(Q) — J(Q) — h{Q). Furthermore, for each Q e A^i,i^('-f), I{Q) 
admits the representation 

j^Q. I H{Q\P{n{Q))) ifQo7T^^=„ 
[ +0O otherwise. 

Remarks: This is a mild generalization of [24, Th. 2.1]. First, as in [24], 
the statement is stronger than usual statements pertaining to the topology of 
weak convergence since the topology a-{Aii.y^{X), Cip{X)) is finer. Second, while 
[24] studied Zn = zq for some fixed zq so that i^n ~ S^o, we need to consider 
more general starting points for each particle, with the only proviso that the 
initial empirical measures i^n converge weakly to v. This generalization also goes 
beyond the chaotic initial conditions considered in [10], [19], and [14]. Third, 
there is another difference with [24] in that not all transitions arc allowed, but 
only those in £". To get the same results, irreducibility of the directed graph 
G{Z,£) suffices. 

Proof. This is a straightforward extension of the proof of [24, Th. 2.1]. The 
arguments needed for the extension are highlighted in Section 3 for the sake of 
completeness. D 

2.3. Empirical Process 

Recall the mapping 

IN ■■ (xn, l<n<N)e Di[0,TlZ^) ^ (In : [0,T] ^ MiiZ). 

The flow /ijv takes values in the space D{[0, T],A4i{Z)). Equip this space with 
the metric 

PT(e,r)= sup po(6,^*), e,e'ei?([o,r],Xi(z)), (2.12) 

0<t<T 

where po is taken to be a metric on Adi{Z) that metrizes the topology of weak 
convergence on M.i{Z). The space D{[Q,T],Mi{Z)) with the metric pT is not 
separable. We are now interested in the law p^^ of p.^ which is the push forward 
PvN = Pj-jv o 77v^- Since hn = 7r(Afjv), we can also write pi-w as the push 
forward pi J = Pi„ ' o tt^^ 

Lemma 2.1. The mapping tt : M.i.ip{X) — > D{[{),T]^ AAi{Z)) is continuous at 
each Q G M.i^ip{X) where J{Q) < +oo. In particular, for each t € [0,r], the 
projection iTt : A4i^ip{X) — > A4i{Z) is continuous at each Q G A4i^ip{X) where 
J{Q) < +O0. 

Proof. The proof is included immediately after Lemma 3.8 in Section 3.2. D 
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To give a precise description of the rate function associated with the large 
deviation principle for the empirical process, some additional notation is needed. 
We once again follow [24]. 

Define t : K ^> M+ to be 

t{u) = e" - u - 1, 

and let t* : K. ^^ M+ be its Legendre conjugate 

(m + l)log(u+ 1) - u ifu>-l 

T*{u) = { 1 ifM = -l 

-|-oo if w < —1. 

For 6* : Z ^ E, ^ e Mi{Z), define 

111^1115= sup lY^omii)- Y. ^(*(j)-<i'WK«A,,j(o 

Recall the infinitesimal generator A^ in (2.2). We can view $ : Z — > M and A^^ 
as column vectors and view A^ as the rate matrix 

of transition rates, where the entries are taken to be zero if j ^ ZiU {i}. The 
adjoint of this (matrix multiplication) operator is of course its transpose At. 

The rate function of interest will be the following. For a i^ G A4i{Z) and 
HeD{[0,T],MiiZ)), define 

g r /[o^j,] lllA(i) --4*(j)/x(t)|||^(t) dt if /^(O) = i^ and Ai e ^ 

' ' ' \ +O0 otherwise, 

(2.13) 
where A is the set of absolutely continuous functions on [0, T]. 

Theorem 2.2. (a) Suppose that the initial conditions vn ^ t' weakly. Then 
the sequence {pi^,^ ,N > 1) satisfies the large deviation principle in the space 
D{[0, T], A4i{Z)) (under the topology induced by the metric pr, see (2.12)) with 
speed N and good rate function S'rg yi(/i|j/). 

(h) If the path fi G D{[0,T], A4i{Z)) has S'[o^t](mI'^) < +oo, then ^j, E A and 
there exist rates (li_j{t),t G [0,r], {i,j) G £) such that 

• fi(t) = L(t)* flit) where L{t) is the rate matrix associated with the time- 
varying rates {li,j{t), {i,j) G £) and L{t)* is its adjoint; 

• the good rate function S[q^t]{M^) "^^ given by 

%t](mI^)- / \ E i^^it)im^AKt))r*( .^'fl,, -l)] dt. (2.14) 

J[0,T] L^^^^ \\ij[fJ.{t)) )\ 
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(c) Suppose that the following hold: fi Cz A, fi{0) = ly, there exist time-varying 
rates {lij{t),t £ [0,r], (i,j) G £) such that the associated rate matrix L{t) 
satisfies fl(t) = L(t)* ii(t), and the right-hand side of (2.14) is finite. Then the 
good rate function S[Q^T]ilJ'\i^) evaluated at ^ is given by (2.14)- 

Proof. With the generahzation available in Theorem 2.1, the proof (of all three 
statements) is identical to the proof of [24, Th. 3.1]. Sec also [15, Th. 2] for 
statements on the nature of the rate function. D 

Remarks: 1. Yet again, this is a mild generalization of [24, Th. 3.1], and of 
the results in [10], [20], and [14] that assume chaotic initial conditions. We allow 
any arbitrary sequence of initial conditions i/jv so long as vn — > v weakly. See a 
consequence in Corollary 2.1 below. 

2. For any initial measure v, the cost S'[o,t](mI^) of the limiting McKean- 
Vlasov path with the initial condition v is 0. This is the path ji : [0, T] — !> A^i(Z) 
that satisfies the ordinary differential equation (ODE) 

/i(i) ^ A;(,)/i(t) (2.15) 

with initial condition /x(0) = u. See (2.13). 

3. Also observe from (2.14) that if the rate function S\q_t]{iA'^) — then ji 
must be the unique solution to the McKean-Vlasov equation /i(i) = A* i^-.^{t) 
with initial condition /^(O) = v. (That the solution is unique follows from Lip- 
schitz assumption in Assumption (A) which implies the well-posedness of the 
ODE). The claim pi„-^ — > 5^(.) follows. 

4. When a path jjl is such that S'[o^t](mI'^) < +oo, the first bullet in the second 
statement of Theorem 2.2 says that there is a control (tilt), given by the rate 
matrix L{t), such that the normal limiting trajectory under this control is /i. 

5. For any finite B >Q and any u £ Mi{Z), the set {^ £ A\ 5[o,t] if^W) < B} 
is compact since S[o^T]i'\i^) is a good rate function. 

The following is a straightforward corollary which shows that S[q^t] il^W) is a 
rate function for a large deviations principle that holds uniformly in the initial 
point. 

Corollary 2.1. For any compact set K C A4i{Z), any closed set 

F C D{[0,T],Mi{Z)), and any open set G C D([0, T],7Wi(Z)), we have 

limsup — logsupp(,^'{AiAr e i^} < - inf S'[o,T](Aik), (2.16) 



Af-i-oo 



N ueK yeK,t,eF 



liminf 1 log inf p(,^) {mat £ G) > - sup inf 5[o,T](/^k). (2.17) 

Proof. This is immediate from [13, Cor. 5.6.15] in conjunction with Theorem 
2.2. D 



2.4- Empirical measure at initial and terminal times 

We first study the behavior of the empirical measure at terminal time alone. 
For T > 0, recall that the law of the empirical flow hn with initial condition 



Borkar and Sundaresan/Invariant Measure in Mean Field Models 13 

MAf(O) = I'M (arising from z^) is ply„ = P „ ° tt"^. The empirical measure 

at terminal time T is ^n{T); its law is the push forward p^ rp = P^„-^ o ttj^ . 
The following result is an easy consequence of the contraction principle [13, Th. 
4.2.1]. 

Theorem 2.3. Suppose that the initial conditions viq — > v weakly. Then the 
family {p\^J'j'^ N >1) satisfies the large deviation principle in M.\{Z) with speed 
N and good rate Junction 

5T(ek) = infl^icTiCMli^) I m = ^,A^(r) =c,/^ e A}. (2.18) 

Furthermore St{0\^) is hounded for all v,^ (z A4i{Z), and we may restrict 
attention in the infimum to fi £ A that also satisfy Sm^TlifJ-W) < +oo. Moreover, 
the infimum is attained, and there exist Ji G A with 71(0) = v, Ti{T) = ^, and 
rates {li_j{t),t G [0,r], (i,j) G £) with associated rate matrix L{t) such that 
'p{t) = L{t)*'p{t), Sri^lv) ~ S\i^j^-\{jl\v), and S'[o.t](mI'^) satisfies (2.14). 

Proof Recall that the metric on D{[0,T],Mi{Z)) is pr given in (2.12). It fol- 
lows that the mapping ^ e D{[0,T],Mi{Z)) i-> ^(T) G Mi{Z) is continuous. 
The statement on the validity of the large deviation principle with good rate 
function (2.18) follows by the contraction principle [13, Th. 4.2.1]. In the sec- 
ond statement of the next lemma, we show that St{^W) is bounded for all 
v,S, G Mi{Z). The goodness of the rate function S'[o^t](mI^) implies that the 
infimum in (2.18) is attained. The rest follow from Theorem 2.2. D 

Let us now prove the boundedncss and related properties of St{^\i^)- 

Lemma 2.2. The following statements hold. 

• There exists a constant Ci{T) < +oo such that for any £,,v G A4i{Z), 
there is a piecewise linear and continuous path fi having constant velocity 
in each linear segment and S'[o,r](A'l^) ^ C'i(T). 

• For any S,,v € M-i{Z), we have ST{i,\v) < Ci(T). 

• There exists a constant C2 < +00 such that for every e > 0, there is a 
6 S (0,e) such that po{iy,C) < S implies Se{^\v) < C2£. 



Proof. The main idea is to show that the difficulty of passage near the neigh- 
borhood of a constant velocity straight line path is bounded. See Section 4. D 

We next have a useful uniform continuity result. 

Lemma 2.3. The mapping (ly,^) >-^ S't(CI^) is uniformly continuous. 

Proof. See Section 5. □ 

Thus far, the only condition we imposed on the initial conditions were that 
z^jv — ^ '^ weakly. Let pj, denote the law of the initial empirical measure /^Ar(0) 
and let p^ J denote the joint law of (^a? (0) , fiN (T) ) . We now consider (pg j, , A^ > 

!)■ 
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Theorem 2.4. Suppose that the sequence (pQ , N > 1) satisfies the large devi- 
ation principle with speed N and good rate function s : M.i{Z) — !■ [0, +oo]. Then 
the sequence of joint laws {p\^ ji , N > 1) satisfies the large deviation principle 
with speed N and good rate function 

Proof. Sec Section 6. D 

Before we elose this section, let us briefly consider arbitrary but finite time 
durations. Define 

]/(e|z^):=inf{5[o,T](A^k) | /i(0) = i., Ai(r) = ^, t £ [0, T], T > 0}, (2.19) 

the infimum cost for traversal from i' to ^ over all finite time durations. This is 
a quantity analogous to Sri^l^) with the same property of uniform continuity. 

Lemma 2.4. The mapping (i^, ^) t-^> Vl^lv) is uniformly continuous. 

Proof. See the last part of Section 5. D 

2.5. Invariant measure: A control theoretic approach 

We now provide some preliminaries for studying the asymptotics of the invariant 
measure within a control theoretic framework. In the next two sections, we will 
state and prove our main results using this approach. 

The directed graph G{Z, E) is irreducible, and so, for each iV, the finite-state 
continuous time Markov chain Mj^ is irreducible. Consequently, it has a unique 
invariant measure, which we call p'^'. In this section, we establish a large 
deviation principle for {p^^\N > 1). We begin by establishing subsequential 
large deviation principles. 

Lemma 2.5. For any sequence of natural numbers going to -|-cx), there exists a 
subsequence {Nk, k > 1) such that {p^^''\ fc > 1) satisfies the large deviation 
principle with speed Nk and a good rate function s that satisfies 

s(n ^ inf [sM + Sri^W)] for every T > 0. (2.20) 

Furthermore, s > and there exists a i>* £ Aii{Z) such that s{i'*) = 0. 

Proof. Since the topology on Aii{Z) with metric po has a countable base, 
and because Aii{Z) is compact, by [13, Lem. 4.1.23], there is a subsequence 
Nk -^ +00 of the given sequence such that (p'^^'=\ Nk > 1) satisfies the large de- 
viation principle with speed Nk and a good rate function s : M.i{Z) — > [0, -l-oo]. 
We now verify (2.20). Fix an arbitrary T > 0. By Theorem 2.4, with pQ = 

p^^\ the invariant measure, the sequence of joint laws (pg ^ , A; > 1) satisfies 
the large deviation principle along the subsequence (iVfc, fc > 1) with speed Nk 
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and good rate function S'o,t(i^, C) — •s(^) + Sri^li^)- By the contraction princi- 
ple, the sequence of terminal laws (p^ '° , fc > 1) satisfies the large deviation 
principle along the subsequence with the good rate function 



inf So,t{'^,0= inf Hiy) + Sri^W)]. (2.21) 

But p^^) is invariant to shifts which yields Px = Po = P^^''- The infimum on 
the left-hand side of (2.21) must therefore evaluate to s(^), which yields (2.20). 
Rate functions are nonnegative and have infimum value of 0, i.e., s > and 
vni^eMiiZ) s{v) = 0. Since s is a good rate function, the infimum is attained 
at some point; call it u* . The proof is now complete. D 

There are multiple solutions to (2.20). Indeed, s = is one of them. But the 
rate function that we are after will have to satisfy further conditions. We turn 
now to explore these towards our goal of identifying a unique solution. 

We shall now consider paths that are of time-duration mT that end at ^. 
Since the terminal condition is fixed, it would be convenient to fix the terminal 
time as and look at negative times; in particular, paths in the time interval 
[—mT, 0] for m> \. But then, we may reverse time and consider the dynamical 
system 

'(i{t) = -L{t)*jl{t), A(0)=C, ie[0,mT], m > 1. 

We use hats as in fl,lij, Xij, L to denote quantities where time fiows in the 
opposite direction with reference to the direction under the McKean-Vlasov 
dynamics. In particular. 



m = 


= n{mT - t) 


hAt) - 


- lij{mT — t), for all i,j £ Z 


m - 


= LimT-t). 



for t G [O,™!"]. Also, for uniformity in notation, let Aij(-) = Aij(-) for all (i,j) 
pairs. One then views the L{t) above as the control at time t when the state is 
{i{t) with cost function at time t given by 

f{m,m) - E mm)KAm)r* ( -^^777- - 1 ) • 

Observe that the cost function is zero when ^ij(t) = Xij{p,{t)) for almost every 
t in the time duration of interest. The total cost is 

/ \ E imimAm)r* ( -^^f^ - 1 ) 1 dt. (2.22) 

which is simply Sro, mT] (mI/^(0)) of (2.14) as can be verified by a change of variable 
in (2.22) that takes t to mT - t. 
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The following Icmnia indicates the existence of one optimal path fi of infinite 
duration, starting at ^. This is crucial for fixing the value of s(^) at points other 
than equilibria. 

Lemma 2.6. For each ^ e j\4i{Z), there exists a path jl : [0,+oo) — ?■ Mi{Z) 
and a family of rate matrices {L{t),t G [0, +oo)) such that (jl{-) satisfies the ODE 

fi{t)^-L{t)*ii{t), te[0,+oo) (2.23) 

with initial condition fi{0) — ^, and 



s(C) == s{[i{mT)) + / f(/i(t), L{t)) dt for all m > 1. (2.24) 

J[0,mT] 

Remark: While the McKean-Vlasov dynamics or the more general /((i) = 
L{t)*ii{t) with L{t) being a rate matrix ensures that /i(<) lies within A4i{Z), 
this is not the case for the dynamics given by (2.23). Indeed, at the boundary of 
Aii{Z), viewed as a subset of M'', the velocity for the dynamics in (2.23) points 
towards a direction of immediate exit from J^i{Z). The state space for the 
dynamics of (2.23) is therefore not restricted to A^i(Z). However, the lemma 
assures us that the selected path jj, stays within the compact subset Aii{Z) for 
all time. 

Proof. Our approach to prove this is the following. We shall define a topology 
on a suitable subspace of paths of infinite duration, and then show that we can 
restrict attention to a compact subset. We shall then argue that there exists a 
nested sequence of decreasing compact subsets, each of which is nonempty and 
all of whose elements satisfy the desired properties. The intersection will then 
be nonempty to yield the desired path. 

Step 1: We shall now restrict attention to paths that lie inside M.i[Z). For a 
path jl : [0, +oo) — ?> Aii{Z), define its restrictions to [0, mT] by 

/i ^ Vrn/i(-) = A^™'(-) : [0,mr] -^Mi{Z) 

which is the restriction of the path /t to [0, mT]. Consider the space of paths of 
infinite duration with metric 

oo 

Poo(A> ^) == X! ^^™ (Prnri'ipnifl; 'ipniV) A 1) . 
m— 1 

Obviously, ipm is continuous for each m. 

We shall also consider reversed restrictions (denoted without hats) defined 

by 

li^''''\t)^fi^"'\mT-t)^fi{mT~t), te[0,mT]. (2.25) 

Fix a B £ [0, +cxd), and consider the set 

Too = (a(-) : [0,+^^) -^MiiZ) I sup 5[o.™T](/i^™V^™H0)) < b] . (2.26) 
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Also, with rj{t) — f]{mT — t) for t G [0, mT], define 

r,„ = {7} : [0,77iT] ^ Mi{Z) I S[o,mT]iv\vm < B} . 

Too is compact. To see this, take an arbitrary infinite sequence (/i„, n > 1) C 
Too- Since 5'[o_mT] is a good rate function, using Lemma 5.2 and Lemma 5.1, it 
is easy to see that each Tm is compact, and so one can find an infinite subset 
Vi C N such that (-0i/i„,n S Vi) C Fi converges. Take a further subsequence 
represented by the infinite subset V2 C Vi such that {^p2fin,n G V2) C r2 
converges. Continue this procedure and take the subsequence along the diagonal. 
This subsequence converges for every interval [0,mT]. For each t define p,(t) to 
be the point- wise limit. Since for each m, we have ipmfj' S F^, it follows that with 
/z(") defined as in (2.25), sup„>i S'[o,™t](m^™V^"'^(0)) < B, and so /t 6 Foo- 
Thus Foo is sequentially compact, and by virtue of its being a subset of a metric 
space, Foo is compact. 

Fix ^ G Adi{Z). Consider the time duration [0,T]. We know from Lemma 
2.5 that there is a v* with s{v*) = 0. Using this in (2.20) of Lemma 2.5, we get 

s(C)<5T(^k*)<Ci(r) 

where the last inequality is due to the second statement in Lemma 2.2. Take 
the constant B = Ci(T); the corresponding Foo is compact. 

Step 2: Observe that (2.20) can be viewed as a minimization over path space, 
with paths fj. of duration [0, mT] ending at ^. Starting from any initial location 
u, the minimum value is upper bounded by B. Indeed, traverse the McKean- 
Vlasov path with initial condition 1/ for duration {m~l)T. This contributes zero 
to the cost. Then proceed to ^ in T units of time. This costs at most B = C\{T). 
The minimum cost to go from 1/ to ^ in time [0, mT] is thus at most B. If we 
consider reversed and translated time so that initial time is 0, the reversed paths 
fi begin at ^ at time 0, have cost at most B, and stay in A4i{Z) for the duration 
[0,mT]. 

Let 



m 



U {fi I ^iit) = f,{mT - t), ^l{0) = u, ii{mT) = £,, 



i.e., the collection of all minimum cost paths jl in [0, mT] from ^ to every location 
in Aii{Z); the minimum cost is at most B. Clearly, F^ C F,„, and F*„ ^ 0, m > 
1. 

FJj^ is also compact. This set being a subset of the compact set F™, it suffices 
to show that F*„ is closed. Let fi he a point of closure of F*„. We can then find 
a sequence {fiS^\k > 1) C V^ such that linife^+oo A''"' = A- Clearly, we must 
have A(0) ~ ^- Let v ~ fi{mT). By a simple application of lower semicontinuity 
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of 5'[o,mT]("|'^): Lemma 5.2. and Lcmm.a 5.1, wc m.ust have 

S[o.mTM'y) < liminfVmT](M^''V^'HO)) 

A;— J-+00 

= liminf5,„T(?|/i^'^(0)) 

A;— > + oo 

where the last inequahty follows because of the continuity of SmT in its argu- 
ments. But S'mT(Cl^) is the least cost for paths that traverse from i^ to ^ in 
duration [0,mT]. So we must have S'[o,mT](/^|'^) = 5'mT(^|i'), which establishes 
that fi e r*„; r*„ is therefore closed. 

Step 3: Let us now finish the proof of Lemma 2.6. Since F^ is nonempty 
and compact, the continuity of ipm implies that ip^^T*^ is nonempty and closed. 
Further, being a closed subset of the compact set Foo, "(Am^r^ is itself compact. 
A simple dynamic programming argument further shows that '0~^F^ is a nested 
decreasing sequence of subsets. Their intersection is nonempty. Take a /t in the 
intersection. 

Focusing on the duration [mT,mT + T], the path t G [0,T] i-^ r]'^"'\t) = 
fl{mT + T -t) has 5[o,T](??^"^h^"H0)) < B < +oo, and so, by the last part 
of Theorem 2.2, we can find rates L^™-\t) such that 77'™) satisfies f]^"^\t) = 
(L(™)(i))*77('")(i), with initial condition 77('")(0). Put these pieces of duration 
T together by defining LirnT + t) = L^^'^T - t) for t S [0,T], and we get a 
rate matrix L(-) defined on [0, +00) such that ft is the solution to (2.23) with 
initial condition /i(0) = £,. The last equality (2.24) follows because fi attains the 
minimum in (2.20) for each duration [0,mT]. D 

The next lemma says that the optimal path above must end up in a spe- 
cific invariant set for the dynamics given by the time-reversed McKean-Vlasov 
equation. 

Lemma 2.7. Consider the trajectory jl given by Lemma 2.6. Its uj-limit set, 
which is the set of its limit points as t ^ -|-oo given by 



n^ f]{(i{t'),t' >t}, 



t>0 

is positively invariant for the ODE 

m^-Al^t)m. i>0. (2.27) 

Proof. Take the fi given by Lemma 2.6. It remains within A^i(2'), and satisfies 
the dynamics ji{t) — —L{t)* jl{t) for t G [0, +00) with initial condition /i(0) = ^. 
Furthermore, (2.24) holds (for all in > 1). Since the integrand in (2.24) is 
nonnegative, s{fL{mT)) must decrease as m increases. But s is bounded between 
[0, Ci(T)], and so there is an s* such that s{fi{mT)) Is* as m t -l-oo. 

Consider any arbitrary subsequence of {ll{mT),m > 1) and take a subse- 
quential limit ^'. On this subsequence, take a further subsequential limit of 
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(fi{mT + T), m > 1) and call it v. Call the subsequence (mfc, k > 1), and con- 
sider the paths 

which are of duration T and time reversals of fragments of p.. We thus have the 
subsequential convergence 

(//(""=) (0), Ai™" (T)) ^ (ly, O as fc ^ +00. (2.28) 

Taking limits as fc — ?• +00 in (2.24) and using the fact that s(/i'™'^^(0)) as well 
as s(/i(™'=^(r)) converge to s* as k -^ +00, the integral term, which is easily 
seen to be S'[o,t](m^"V^"^(0)), satisfies 

limsupS'[o,T](yU^""'V^'"'H0)) = 0. 

This fact and the nonnegativity of St imply 

lim 5T(/i^™'^-HT^)lM^™'H0)) = 0. 

By the uniform continuity of St in both its arguments (Lemma 2.3) and by 
(2.28), we deduce that STiCli^) = 0- But then the path that goes from ly to £,' 
in time [0,T] and attains St{£,'W) = is the McKean-Vlasov path which is the 
solution to the dynamics 

/i(i) = ^;(t)M(t), te[o,T] 

with rate matrix ^^(t), initial condition /i(0) = v, and final condition /i(r) = ^'. 
But this implies that 'Jl{t) = ^{T — t) satisfies (2.23) with L{t) = A-p^t) and initial 
condition ^1(0) = ^'. It follows that il, the w-limit set for /i, is contained in the uj- 
limit set for the dynamics (2.27) with initial condition /},(0) = ^. This concludes 
the proof. D 

2.6. Invariant measure: Unique globally asymptotically stable 
equilibrium 

In this subsection, we consider the case when there is a unique globally asymp- 
totically stable equilibrium ^o- 

Lemma 2.8. // the McKean-Vlasov equation (lit) ~ A*(^^fi(t) has a unique 
globally asymptotically stable equilibrium ^q, then s(^o) ~ 0. 

Proof. Consider the McKean-Vlasov dynamics 

m = A;^,^fi{t) 

with initial condition /i(0) = v* . By our assumption that ^o is the unique globally 
asymptotically stable equilibrium, n(T) ^ ^o as T — > +cx). By the second 
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remark following Theorem 2.2, the McKean-Vlasov path has zero cost and so 
STifJ-{T)\v*) = for each T > 0. By (2.20), we then have 

s{^^{T)) < sii.*) + ST{KT)y) = <^*)- 
Take limits as T — > +cx} and use the lower semicontinuity of s to get 

< s(^o) < liminf s(/i(T)) < s(i/*) = 0, 

T— >+oo 



whence s(^o) = 0. 



D 



The following lemma shows that the rate function for any subsequential large 
deviation principle is unique. 

Lemma 2.9. // the McKean-Vlasov equation /i(t) = A* ,^sii{t) has a unique 
globally asymptotically stable equilibrium ^q, then the solution to (2.20) and 
(2.24) is unique and is given by 









dt (2.29) 



where the infimum is over all fi that are solutions to the dynamical system 
fl{t) = -~L{t)* fl{t) for some family of rate matrices L(-), with initial condition 
A(0) — C' terminal condition limt_i._|_oo A(0 ~ ^0. o,iT-d p-{t) G A4i{Z) for all 
t>0. 

Proof. The path given by Lemma 2.6 converges as shown in Lemma 2.7 to fi, 
which is contained in an w-limit set of the ODE (2.27). So il is connected and 
compact. Since the path /i(-) also stays completely within M.\{Z) (viewed as 
a subset of R''), J7 is a subset of M.\[Z). Furthermore, il is invariant (both 
positively and negatively) to the dynamics defined by the ODE (2.27). But 
then, by our assumption that ^0 is the unique globally asymptotically stable 
equilibrium for the McKean-Vlasov dynamics 



A(0 = ^;(t)M(i), 



(2.30) 



we must have J7 = {^0} because this is the only subset of A^i(Z) that is invariant 
to both of the dynamics in (2.27) and (2.30). Letting m — > +00 in (2.24), it 
follows by lower semicontinuity of s that 



m > s(Co) 



[0, + oo) 



[ E imimuimy* 



(»j)e£ 



Xijifi{t)) 



By Lemma 2.8, s(Co) = 0, and thus 



dt. 
(2.31) 



siO> 



[o,+oo; 



[ E mm)k,im)^ 



(»j)e£ 



k,j{fi{t)) 



dt 
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for the special path ji. This cstabhshes that s(^) is at least the right-hand side 
of (2.29). By an application of (2.20) with v = ^o, it is obvious that s(^) is upper 
bounded by the right-hand side of (2.29), whence equality holds in (2.29) and 
the uniqueness of s(^) follows. D 

We are now ready to state and prove our result on the sequence of invariant 
measures. 

Theorem 2.5. Let the McKean-Vlasov equation fi{t) ~ A* ,^-.fi{t) have a unique 

globally asymptotically stable equilibrium ^o- Then the sequence {p^^\N > 1) 
satisfies the large deviation principle with speed N and good rate function s given 
by 

'io = inf / [ E imimAmy* ( ,^''f-\, -i]]dt (2.32) 
A" j[o,+oo) \,%£ vA,,,(/i(t)) ; J 

where the infimum is over all jl that are solutions to the dynamical system 
fl{t) — -~L{t)* fl{t) for some family of rate matrices L(-), with initial condition 
A(0) — C' terminal condition limt_>.+oo A(0 ~ ^o> o,nd p,{t) G A4i{Z) for all 
t>0. 

Proof. Take any arbitrary sequence of natural numbers growing to -I-cxd. By 
Lemma 2.5, there is a subsequence that satisfies the large deviation principle 
with rate function s such that (2.20) holds. Given our assumption that ^o is 
the unique globally asymptotically stable equilibrium for the McKean-Vlasov 
equation, we have s > 0, and s(^o) = by Lemma 2.8. By Lemma 2.9, s is 
uniquely specified by (2.29), which is the same as (2.32). Thus every sequence 
contains a further subsequence {Nk, k > 1) such that {p^^''\ k > 1) satisfies the 
large deviation principle with speed Nk and the same rate function s specified 
by (2.32). By [13, Ex. 4.4.15(a)-(b), pp. 147-148], it follows that {p^^\ N > 1) 
satisfies the large deviation principle with speed N and rate function given by 
(2.32). D 

Theorem 2.5 provides a complete characterization of the rate function. How- 
ever, numerical computation of the rate function is a challenging problem. One 
might possible discretize time and the state space, and employ dynamic pro- 
gramming techniques to get an approximation. This is an interesting line of 
work that is beyond the scope of this paper. See [4] and references therein for 
some results based on exit times. 



2.7. Invariant measure: The general case 

We now treat the general case. Recall the definition of V in (2.19). Let us define 
an equivalence relation on Mi{Z) as follows. We say i^ ^ (^ ii V{£,\i^) = V{iy\£_) = 
0. Using the third part of Lemma 2.2, it is easy to show that the set of points 
that are equivalent to each other is closed and therefore compact (being a closed 
subset of the compact set Mi{Z)). 
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Wc will generalize Theorem 2.5 under the following assumption on the dy- 
namical system corresponding to the McKean-Vlasov equation (2.15): 

(B) There exist a finite number of compact sets Ki,K2, . . . ,Ki such that 

1 . i^i , J^2 G Ki implies vi ^ V2- 

2. z^i e Ki, V2 ^ Ki implies vi no v^- 

3. Every w-limit set of the McKean-Vlasov equation /i(t) — A* ,^.^{t) is 
contained in one of the Ki . 

Under the hypothesis of Theorem 2.5, assumption (B) holds with / = 1, 
Ki = {Co}, and s(Co) = by Lemma 2.8. We now generalize Lemma 2.8. 

Lemma 2.10. Under assumption (B), the rate function s satisfies the following: 

• There exists fo £ ^io Z'^'' some ig = 1,2,...,/ which satisfies s{^q) ~ 0. 

• There exist nonnegative real numbers si, S2, ■ ■ ■ , si such that ^ £ Ki implies 
s(0 = s.- 

Proof. The first statement immediately follows from the steps in the proof of 
Lemma 2.8: ^o is now some clement in the w-limit set for the McKean-Vlasov 
dynamics with initial condition v* satisfying s^v*) = 0. 

For the second statement, let j^, ^ G Ki. Fix e > 0. Since i^ ^ £,, there exists 
T > such that St{CW) < ^- Using (2.20), we get 

s{0 < si'^) + St{^W) < s{i^) + e. 

Reversing the role of ^ and i', we get s(i^) < s(C) + e, whence |s(C) — x{i')\ < e. 
Since e was arbitrary, we must have s{i') = s{^). So all points in a compact set 
Ki take the same value. The second statement follows. D 

We now argue that the function s satisfying (2.20), (2.24), s > 0, and 
min^ s(i^) = is indeed unique. Wc do this in two steps via the following lemmas. 

Lemma 2.11. Assume that (B) holds. Let Si, 82, . ■ . , si be specified as the val- 
ues on the compact sets Ki, K2, . ■ . , Ki. Let si' > for 1 < I' < I and let 
min{si, S2, . . . , si} = 0. Then the solution to (2.20) and (2.24) is unique and is 
given by 



s(C) = inf inf 
I' fi 



Jo '- ,: -.^^ \ Ai ,(U t 



'0 (..j)e£ ^ \A,j(A(0) 

(2.33) 
where the second infimum is over all fi that are solutions to the dynamical system 
fl{t) — —L{t)*fl(t) for some family of rate matrices L{-), with initial condition 
A(0) — ^! terminal condition fi(t) — !► Kii as t ^ +00, and fi(t) S A4i{Z) for all 
t> 0. 

Proof. The same steps of the proof of Lemma 2.9 apply with the following 
modifications. J7 = {^o} gets replaced by fi C Kii for some I'. Consequently, in 
the lower bound in (2.31), s(Co) gets replaced by Sf. D 
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In order to specify the solution for si, . . . , sj, we now define tlie following. 

V{K,,K,) := mfj5[o,T](A^|/i(0)) I 

m(0) e K,,^l{T) e A'j,/^(t) i li^'^^JK,' for t G [0,1^.^4) 
If the set is empty, the infiniuin is taken to be +00. Let us also define 

ViK„K,) := V^(ek)LeK.,5eK, . (2.35) 

By Lemma 2.10, the value is independent of i/ G Ki and ^ G Kj. 

As indicated by Freidlin and Wcntzell ([21, p. 171]), using Lemma A. 2 it is 
easy to argue that 

n 
A mm[V{K„ K,, ) + ViK,, , K,., ) + y(i^„ , K^)] 

41,42 

A---A min [V{K„ K,,) + ■ ■ ■ + ViK,, ^, K,)]. 

4l,...,I,_2 

Consider the indices {1,2,...,/} for the compact sets Ki,K2, ■ ■ ■ ,Ki. Let 
G{«} be the set of all directed graphs on the vertex set {1, 2, . . . , /} such that 

• there is no outward edge from i; 

• a vertex j 7^ i has exactly one outward edge; 

• there are no closed cycles in the graph. 

Define 

WiK,,)^ min y V{Iu,Kj), (2.36) 

and finally 

s,, = W{K,:)-umYW{K,). (2.37) 

i 

Lemma 2.12. The rate Junction s has values si, S2, ■ ■ ■ , si given by (2.37). 
Proof. Immediate from Theorem A.l in Appendix. D 

Remark: In order to obtain these values, one has to go beyond the ODE 
method. One has to consider the empirical measure Markov process sampled at 
hitting times of neighborhoods of the compact sets. This is done in Freidlin and 
Wcntzell [21, Ch. 6] for diffusions on a compact manifold, with V satisfying a 
Lipschitz property. Thanks to Corollary 2.1, Lemma 2.2, Lemma 2.3, the same 
program can be carried out with straightforward modifications to account for the 
fact that we have to handle jumps and the fact that the minimum cost function 
y(-|-) satisfies only the uniform continuity property. The appendix provides the 
necessary verification. 

With this disambiguation for the values of s at the compact sets Ki , we now 
have the following generalization to Theorem 2.5. 
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Theorem 2.6. Assume that (B) holds. The sequence {p'-^\N > 1) satisfies 
the large deviation principle with speed N and good rate function s given by 



s(^) = inf inf 
I' fi 



+ 00 

' ^^ " ' ' dt 



[ Y, imimAm)^ 



hAt) 1 



'0 {^,J)e£ ^ XhjiKt)) 

(2.38) 
where the second infimum is over all fi that are solutions to the dynamical system 
fl{t) = —L{t)* fl{t) for some family of rate matrices L{-), with initial condition 
fl(0) = ^, terminal condition fl(t,) — >■ Kii as t ^ +oo, and (i{t) G A4i{Z) for all 
t>0. 

Proof. Same as that of Theorem 2.5, but with the use of Lemmas 2.11 and 
2.12. D 

The rest of the paper gives the remaining proofs that corroborate the state- 
ments in this section. 



3. Proof of Theorem 2.1 

The proof is based on a generahzation of Sanov's theorem due to Dawson and 
Gartner [11], the Girsanov transformation, and the Laplace- Varadhan principle. 
We proceed through a sequence of lemmas. 

3.1. The noninteracting case 

Consider first the noninteracting case. 

Lemma 3.1. Suppose that the initial conditions vn — > i' weakly. Then the 
sequence I P°jv ,N> 1) satisfies the large deviation principle in Aii,ip{X), 

endowed with the weak* topology a{Mi^^{X),C^{X)), with speed N and good 
rate function J{Q) given by (2.10). 

Proof. The family {Pz,z £ Z} is clearly a subset of A^i_^(A') since the transition 
rate from any state to any of the other (at most r) states is upper bounded by 1. 
The family {Pz, z G Z} is also Feller continuous in the discrete topology on Z. 
Since v^ — ^ v, by Dawson and Gartner's [11, Th. 3.5] which is a generalization 
of Sanov's theorem, we have that (P°jv \N > 1) satisfies the large deviation 
principle in the weak* topology a{M.i^ip{X),C^{X)) with speed N and good 
rate function J{Q) given by (2.10). D 

Our next lemma states that A^i,i^(-^) contains all the probability measures 
of interest to us. 

Lemma 3.2. // J{Q) < +oo then (1) Q e Mi,^{X) and (2) Q o n^^ = ly. 
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Proof. Observe that \\(p\\tp < 1, and the topology on X is chosen so that (p is 
continuous. Hence (p £ C^{X). Recalhng the expression for J{Q) in (2.10), we 
get that J{Q) < +oo implies 



[ pdQ-Y] v[z) log / ey dP, < 
Jx ^^2 ■'^ 



(3.1) 



Since the transition rates for P^ are upper bounded by 1, and there are at most r 
possibilities for jumps from any state, (/3 is stochastically dominated by a Poisson 
random variable with parameter rT. Consequently 1 < ^•^e'^ dPz < +oo for 
each z G Z, and it follows from (3.1) that /^ (p dQ < +oo, and so Q G Aii.,p{X). 
To prove the second conclusion, suppose vq = Qottq ^ v. Consider bounded 
functions f{x) = fQ{TTa{x)) that depend on x only through the initial condition. 
Since vq ^ i>, there exists a function / of the above form that also satisfies 
J2z fo{z)iyQ{z) - J2z h{z)v{''^) ^ 0- By fiipping the sign of / if necessary and by 
scaling, we may assume that ^^ fQ{z)vQ{z) — X^z fo{'^)^{^) — ^ for an arbitrary 
a G (0,+oo). This / is bounded continuous and hence is in C^{X). A simple 
calculation yields 

[ fdQ-Y, ^i^^) log / e^ dPz = Y. hi^)M^) ~ E M^M^) = «■ 

Since a > was arbitrary, J{Q) — +cxd, and the second part is proved by 
contraposition. D 

We next get an alternative expression for J{Q) as a supremum over the more 
familiar space Cb{X) of bounded continuous functions on X . 

Lemma 3.3. For each Q E Mi,^{X), J{Q) defined in (2.10) admits the alter- 
native characterization 



JiQ) 



sup 



f dQ 



X 



zez 



v{z) log / eJ dP; 



X 



(3.2) 



Proof. In (3.2), the supremum is taken over Cb{X), the set of bounded contin- 
uous functions on X, while in (2.10) the supremum is over C^{X). 

Fix Q G Mi,^{X). Since Cb{X) C C^{X), J{Q) defined by (2.10) is at 
least the right-hand side of (3.2). For the other direction, let / G C^{X), and 
consider the truncations /„ of the function / to [— n,n]. Clearly {/„} C Cb{X), 
and /n — > / a.e.-[(3] and a.e.-[P2]. Since we also have J^ f dQ < +oo and 
J^ e^ dPz < -|-oo, both of which can be easily checked using \\f\\ip < -f-oo, an 
application of the Lebesgue dominated convergence theorem yields 



lim 

n— > + oo 



/ .fn dQ-V] v{z) log / e-^" dPz 
Jx ^g2 ■'^ 

f f dQ-y^v{z)\og I ef dPz. 
Jx „^., Jx 
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Since / g C^{X) was arbitrary, J{Q) in (2.10) is at most the right-hand side of 
(3.2). D 

We now characterize J{Q) further. We begin by getting a lower bound. 

Lemma 3.4. Let Q be such that Q o ttq"^ = i>. Define P as in (2.7). We then 
have H{Q\P) < J{Q). 

Proof. For any / G Cb{X) and with P as defined in (2.7), Jensen's inequahty 
yields 

f fdQ-Y] Hz) log / e^' dP,> f fdQ- log / ef dP 
Jx ^ Jx Jx Jx 

By Lemma 3.3, J{Q) is the supremum of the left-hand side over all / G Cb{X). 
By the variational formula for relative entropy ([13, Lem. 6.2.13]), H{Q\P) is 
the supremum of the right-hand side over all / G Cb{X). This establishes the 
inequality. D 

We next show that J{Q) is upper bounded by another relative entropy. To do 
this, let us introduce the Polish space {X,d) where X ^ Z x X and the metric 
d is 

d{{i,x),{j,y)) = l{j^j} +d{x,y), 

where d is the metric on X. The first component of X shall denote the initial 
condition. For two measures Ri and i?2 on X, let H{Ri\R2) be the relative 
entropy of Ri with respect to R2. 

For a fixed v G A^i(Z), let us now define P as 

dP{z,x)=v{z)dPHx). (3.3) 

The push forward of P under the projection mapping (z,a;) G A" i— > a; G <-f is 
clearly the P defined in (2.7). 

Let Q be such that Q oir^"^ = u. We then define Q as 

dQ{z,x) = (iQ(x)l{^Q(^)}(z). (3.4) 

Observing that X is Polish and that the push forward of Q under the projection 
{z,x) t-^ z is v, it follows that there is a regular conditional probability measure 
Q2 satisfying 

dQ{z,x) = v{z)dQz{x). (3-5) 

Putting (3.4) and (3.5) together and summing over z, we obtain that the second 
marginal of Q is 

dQ{x) = ^dQ(x)l{^o(j;)}(2:) = ^v{z)dQz{x). (3.6) 
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Since both Q and P have the same first marginal v, the decomposition result 
for relative entropy [13, Th. D.8] gives 

( yuiz)H{Q,\P,) if Q, < P, a.e.-H 
HiQ\P) = l t^z _ (3.7) 

I +00 otherwise. 

With these preliminaries, we are now ready to upper bound Jiff). 

Lemma 3.5. Let Q be such that QotTq = v. With P and Q as above, we have 

J{Q) < H{Q\P). 

Proof. For / G Cb(A'), by (3.6), we have /^ / dQ = ^^ v{z) J^ / dQz, and so 

/ fdQ-y2,,{z)\og f ef dP, 
J X 2 J X 

= E ^(^) [ / / ^^~' - log / "^-^ '^p^ 

2 U X Jx 

< V v{z) { sup [ / / dQ, - log / ef dp} ) 
^ \f<^Cb(x) Ux Jx \j 

= ii{Q\P). 

where the last equality follows from the observation that the term within paren- 
thesis in the immediately preceding inequality is the variational representation 
for the relative entropy H(Qz\Pz), and from (3.7). Take supremum over all 
/ e Cb{X) and use Lemma 3.3 to deduce that J{Q) is upper bounded by 
HiQ\P). D 

Lemma 3.6. Let Q G M.i.^{X). We then have 

jiQ) = { ^(^1^)' «/Q°V' = ^ 

[ +O0 otherwise. 

Proof. If QoTTj^ ^ z/, by Lemma 3.2, we have J{Q) = +cxd. So assume Qoti'^ = 
V. By Lemmas 3.4 and 3.5, we get 

H{Q\P) < J{Q) < H{Q\P) 

where P and Q are defined in (3.3) and (3.4), respectively. The second marginal 
of P is P = ^^ v{z)Pz. From the definition of P it is clear that dP{z,x) = 
dP{x)l i^gix)}iz) J SO that the regular conditional probability measures of both 
P and Q, given the second component a;, are the same, i.e., dQ{z\x) = dP(z|x) = 
l{7ro(a;)}(-2^) a.e.-[Q]. In particular, H{Q{-\x)\P{-\x)) = a.c.-[(5]. By the decom- 
position result for relative entropy [13, Th. D.8], we get 

H{Q\P) = HiQ\P)+ f H{Q{-\x)\Pi-\x)) dQ{x) 

Jx 
= H[Q\P). 

The lemma follows. D 
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3.2. Continuity of the function h{Q) 



We now proceed to address some preliminaries required for the interacting case. 



The Radon-Nikodym derivative (2.5) of Pi)j^ with respect to P„^ is 



(AT) 



jO,{N) 



dH 



{N) 



HP ''- 



:0,{N) 



(Q) = cxp{7V/i(Q)}. 



Wc now study the continuity property of h{Q). Towards this, we first estabhsh 
a regularity property for all Q with J(Q) < +oo. We then appeal to results of 
[24] to establish the continuity of h(Q) when J{Q) < +oo. 

Lemma 3.7. Let J{Q) < +cxd and suppose that the random variable X is 
distributed according to Q. Then 



sup E 

tG[0,T] 



sup {1{X„5:^X„_}} 

u£[t-a,t+a]n[a,T] 



as a -10. 



(3.8) 



Proof. A proof for the case when Q o ttq = Szg for some fixed zq can be found 
in [24, eqn. (2.14), p. 309-310]. Our argument below is a simple modification 
and relies only on what we have thus far established for J{Q). 

Let K ^ {x € X : 3u € [t — a,t + a]C\ \f),T] satisfying Xu 7^ a;„_}. Since 
J{Q) < +00, it follows that Q <^ P. We may therefore write 



E 



sup {l{x„#x„_}} 

u£lt-a,t+a]nlO,T] 



= Q{K) 



< 



dQ 
X \dP 
dQ 



-^ 1 Ik dP 



dP 



IK 



(3.9) 



T'.P 



where ||/||t-. p is the Orlicz norm 
P = inf i a > 



X 



l./(^)l 



dP{x) < 1 



(with respect to the function r* and measure P), [[(^H^p is a similarly defined 
Orlicz norm with respect to the function t and measure P, and the inequality 
in (3.9) is the Holder inequality in Orlicz spaces. See the Appendix in [24] for a 
summary of key results on Orlicz spaces. 



The lemma's proof will be complete if we can show that 



dQ 
dP 



is bounded, 



IxIIt- p vanishes as a 



T',P 



and 

J{Q) < +00 implies 



dQ 
dP 



0. Wc proceed to justify these claims. 
< +CX). Indeed, since 



r',P 



lim 



T*{U) 



>+cio u log U 



-1, 
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choose a large enough uq > such that t*{u) < 2u\ogu for u > uq. Also 
observe that t*(u) is increasmg m u and it log u > — e^^ for u > 0. Thus with 
/ = dQ/dP, 

I T*{f)dP < T*{uo)+ f 2f log fdP 

Jx J{xi£X:f{x)>Uo} 

< T*{uo)+2J{Q) + 2e-^ 

< +00. (3.10) 

Since r* is convex and t*(0) = 0, Jensen's inequality yields 

T*{.f/a) <T*{f)/aioT a> 1. 

This fact in conjunction with (3.10) implies that ||/||r',p < +oo. 

Now consider ||l/^||^p. Since t(0) = 0, we get T{{lK)/a) = T(l/a)l/<-, and 
so 

J T (^\ dP = T(l/a) j Ik dP = T{l/a)P{K). 

Under P, the transition rates are upper bounded by 1. Moreover, there are at 
most r possible next states. Since K is the event that there is a transition in 
[i - a, t + a] n [0, T], it follows that P{K) < 2ar. From its definition, the Orlicz 
norm is the smallest positive a such that T{l/a)P(K) < 1, and so 

_ 1 1 

""^^"^'■f' " T-^{1/P{K)) - r-i(l/(2ar))' 

where the equality in the above chain holds because t(u) is increasing in u for 
u > 0, and the inequality holds because of the same property for t^^{u). This 
last upper bound vanishes as a J, 0. D 

The following lemma implies Lemma 2.1 as a corollary. 

Lemma 3.8. Consider A4i{D{[0,T], Z)) endowed with the topology of weak 
convergence and D([0,T], A4i{Z)) endowed with the topology induced by the 
metric pr defined in (2.12). Then the mapping 

n : Mi{D{[Q,T],Z)) ^ D{[Q,T],Mi{Z)) 

is continuous at each Q G A^i(-D([0, T], 2)) where J{Q) < +cxd. 

Proof Note that, by Lemma 3.2, J{Q) < +cx) implies Q G Mi,^{X). The 
statement of the above Lemma is the same as [24, Lem. 2.8]. The only difference 
is that our representation for J{Q) differs in order to handle nonchaotic initial 
conditions and allows Q o ttq to be any measure in A4i{Z). 

The proof of [24, Lem. 2.8] holds verbatim if we can establish (3.8) (which 
is the same as [24, eqn. (2.14)]) for the more general case under consideration. 
This is done in Lemma 3.7. D 
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This is an appropriate location to include the proof of Lemma 2.1, which is 
a corollary to the above Lemma. 

Proof of Lemma 2.1. The first part is a corollary to Lemma 3.8. Indeed, Lemma 
3.8 shows that tt is continuous under the coarser topology of weak conver- 
gence of probability measures metrized by dsko- Since the natural embedding 
of X into Z?([0,r],Z) (with topology induced by dsko) is continuous, it im- 
mediately follows that TT is a continuous mapping under the finer topology 

To see the second part, fix a Q such that J{Q) < -l-c«, at G [0,T], and 
consider a sequence Qat -^ Q. By the first part, we have tt{Qn) ^^ '^{Q)- which 
is the same as saying Pt{'^{Qn),'^{Q)) — >• 0. But then 

Po{MQn),MQ)) < PT(7r(Qjv),7r(Q)) ^ 

establishes the continuity of tt^ . D 

We now come to the continuity of the function h{Q). 

Lemma 3.9. Consider the space Mi^tp{X) endowed with the weak* topology 
(^{^i.ipiX), Cip{X)). The function h : M.i^ip{X) -^ M. defined in (2.6) is contin- 
uous at every Q where J{Q) < +cx3. 

Proof. The statement is the same as [24, Lem. 2.9]. The same proof applies. 
That proof requires continuity of tt, which was established in Lemma 3.8, and 
Assumption (A). D 

3.3. The interacting case 

Wc now address the interacting case. 

Proof of Theorem 2.1. Recall the statement of Theorem 2.1. We are now given 
that the sequence of initial empirical measures vn -^ v weakly. By Lemma 
3.1, (P°jv ,N> 1] satisfies the large deviation principle in the topological 
space M.i^ip{X) with rate function J{Q). By Lemma 3.2 and Lemma 3.9, h is 
continuous on the set {Q G Mi^^{X) \ J{Q) < +oo}. Furthermore, by [24, Lem. 
2.10], for every a > 0, we have 



limsupllog / e^-l"! dP;;(^) < 



Using the Laplace- Varadhan principle, see [24. Prop. 2.5], we can draw two 
conclusions. The first conclusion is that 

i-log/ e"''dF;i")-> sup [h{Q')-J{Q')] (3.11) 



as N ^)- +O0. From (2.5), we have 
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a probability measure. The left-hand side in (3.11) is therefore always 0, and so 
supg/g^^ tx)[^iQ')^JiQ')] = 0. The second eonclusion is that iPi)j^,N> 1] 
satisfies the large deviation principle in the topological space Aii.,p{X) with 
good rate function 

HQ) = J{Q) - HQ) - inf [J(Q') - h{Q')] = J(Q) - h{Q) 

where the last equality holds because the infimum above is by the first con- 
clusion. This concludes the proof of the first part of Theorem 2.1. 

We now show (2.11). By Assumption (A), it is easy to see that there exists 
a constant K such that |ft.((3)| < K{1 + J^f ^Q) so that if Q G ■M.i^ip{X) then 
|/i((5)| < +00. By Lemma 3.6, if either Qo-k^^ ^ v ov Q\s not absolutely contin- 
uous with respect to P, then J{Q) = -l-oo, and by the finiteness of h{Q), we have 
HQ) = J{Q) ~ h{Q) = +00. We may therefore assume Qott^^ = v and Q <^ P, 
whence, by Lemma 3.6 once again, J{Q) = H{Q\P) = J dQ \og{dQ/dP). It 
therefore suffices to argue that 

QoTT-^ = iy and Q-^P ^ H{Q\P) - h{Q) = H{Q\P{tt{Q))). 

Let fi = 7r((5) for convenience. Observe that the density 

dP,{t^) 



dP. 



-(•) ^cx.p{hi{-,fi)} 



in (2.3) does not depend on z. It follows that the density of the mixture distri- 
bution P{iJ,) in (2.8) with respect to the mixture P in (2.7) is 

^p (^) = cxp{hi{x,fi)}. 
Using this in (2.6), we get 

dP{^l) 



HQ) = f dQ log ■ 

Jd(10.T].Z) 



lD{[0,T],Z) dP 

from which 

dQ f ,^ 1 , dPi^i) 



H{Q\P)-h{Q) = / dQ log^ - f 

JD(\a.T].Z) O,^ JL 



dQ log ■ 



D{[Vi,T],Z) ^^ Jd{[o,t\,z) dP 

An 1 ^^ 
dQ log- 



'd([[),t].z) dP{n) 

= HiQ\P{^,)) 

follows. This concludes the proof. D 
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4. Proof of Lemma 2.2 

We address the first bullet. For ease of exposition, let us for now allow all 
possible transitions and ignore the constraint that only £ transitions are allowed. 
Consider the constant velocity path 

Mi)= (i-|)i^ + |e, te[0,T] (4.1) 

for which 

A(t) = ^, te[o,T]. 

There is flow out of i if £^{i) > v{i), and flow into i otherwise. We now construct 
a rate matrix L{t) with entries lij{t) that ensure the traversal of this constant 
velocity path. 

Since there is conservation of mass '^^^{z) = X^z'^C^): ^^ '^^^ construct 
mass transport parameters {gij} such that for an i with ^{i) > ^(i) and a j 
with v(j) < ^(j), the quantity gij is the fraction of the excess i^(i) — ^{i) that 
goes from i to j. In particular, {gi.j} satisfies 



g,j G [0,1] foraUi,jeZ 




g,j = a Hi) < m or lyu) > aj) 


(4.2) 


Y^ g,.j = 1 iiv{i)>^{i) 


(4.3) 


r-yiiX^j) 





and finally 



E M*) - eWbu = e(j) - Kj) ifKj)<^(j)- (4.4) 



Equation (4.3) says mass is not destroyed and (4.4) says new mass is not created 
(all mass entering j must come from i's with excesses). 

Define the diagonal elements of the transition rate matrix Lit) to be 

L At) = [ ^WM^ if ^ -^-fi- -(^■) > ^(^■) (4.5) 

1 otherwise. 

Now define the off-diagonal elements of L{t) to be 

I (t)-l ^ ii.ij)<aj),^ez 

'''^' " I -h.j(t)9j, if Hj) > m and * ^ j. ^^'^^ 

We next claim that /t(t) = L{t)* ii{t). Indeed, for i such that v{i) > ^(z), we 
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have 

^=' {fi{t){i))k4t)+ + 

(b) ew-KO 

r 

In the above sequence of equahties, the second term in (a) vanished because 
^0) < ^{j) impUes lj^i{t) = (see (4.6)); the third term vanished because, by 
(4.2) and noticing that i is the second argument, i/(i) > ^(z) impUes gjj ~ 
which in turn impUes Ij^i(t) = again by (4.6). Lastly, (b) follows from (4.5). 
For i such that ^{i) < ^(i), we have 

(i(i)>(t)) {^) 

= E(^w(-?)) ^^-w 

i--3¥=i,i'(j)<(,(j) r-Jiti,'^U)>iU) 

=^0 + 0+ E (mW(j)) (-'«-W,9.vO 

(b) £ 
T 



E ('^O') - ^0'))5..' 



(c) e(»)-KO 
T 

In the above sequence of equalities, (a) follows from (4.5) and (4.6). Equation 
(b) follows from (4.5), and (c) follows from (4.4). The above arguments establish 

m = L{tr^i{t). 

Let us now evaluate the difficulty S\rfx^{^\v) of passage near this constant 
velocity path ji. If we show that the integral in the right-hand side of (2.14) is 
finite, by Theorem 2.2. S'[o.T](A*k) equals this integral. Observe that li,j{t) is 
not bounded if one of /i(0)(i) = v{i) or fi{T){i) ~ ^(i) equals 0, and so wc do 
have some work to do. 

The right-hand side of (2.14) can be expanded to be 

'■•'<" ' (4^7) 



/,jEjM')(0)<..W.og(^,^,„ 



'■j:j#i 



-{^l{t){^))l,^^{t) + {^l{t){i))x,,Ji^lit)) 



dt. 
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From (4.6) and (4.5), wc get (m(0(«)) hjit) = T^^ii^ii) - ^(J))g,j, and this is 
nonzero only if i'{i) > ^{i) and i'{j) < ^(j); see (4.2). For convenience, let us 
define 

T = {i^,J)\J^^, KO>eW, Kj)<e(j)}. 

By Assumption (A), |logAij(-)| < |logC| + |logc|. Using these observations, 
(4.7) is upper bounded by 



[O.T] 






< E (K0-e«)5.,j|iog((K0-e(*))5.j)i 

(ij)eT 

- E H^)-m)T-' f log(/i(t)(z)) dt 

+ ||i/-eili|iogr| 



u^ 



-^||i(|logC| + |logc| + f) + CTr^ 



where in arriving at the last three terms we have repeatedly used (4.3). The 
quantity ||i^ — Clli is the total variation distance between ^ and v. Let us now 
bound the first two terms on the right-hand side of (4.8). 

Observing that there is a constant K such that sup^grp ij a;| logccj < K < +C!0, 
the first term on the right-hand side of (4.8) can be upper bounded as 

E (K0-c«).9.,,iiog((K0"ewK,)i 

< E KO-eW)|log(KO-^«)l( E 9^,J 

< E M*)-c«)|iog(KO-e(*))i + A-||i^-eiii 

i:u{l)>^{i) 

< Ei(iK*)-^«iiogi'^w-^wi)i + ^ii^-^iii- (4-9) 

i 

To bound the second term on the right-hand side of (4.8), use (4.f) and 
employ the change of variable u = l^{t){i) to get 

-{u{i) - ai))T-' I \og{n{i){i)) dt^ f ' \ogu du 

= [lilogW-M]^[*] 

< \m iogc(o - ^(0 iog^(*)i + w) - m\- 
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Summing this over all i, we see that the second term in (4.8) is upper bounded 

by 

El^Wlog^W-'^Wlog'^WI + ll'^-^lli- (4.10) 

i 

Since Aii{Z) is compact, all terms in the upper bounds (4.9) and (4.10) are 
bounded. Substituting (4.9) and (4.10) on the right-hand side of (4.8), and notic- 
ing that T > 0, we see that the right-hand side of (4.8) is upper bounded, and 
this upper bound serves as an upper bound on SlQx]i^J■\l^), which we summarize 
as 

5[o.r](Mk) < 5^|(|^(^)-?(^)|log|K^)-C(^)l)l 

i 

+ J2\^{i)\og^{i) - i^ii)\ogi^ii)\ 

i 

+ \\v - S,M\\ogT\) 

+ \\v-S,\\i{\\ogC\ + \\ogc\+K + 2) + CTr^ (4.11) 
< C3(T) 

for a suitable constant C^{T) that is independent of v and ^. This concludes the 
proof of the first bullet for the case when all transitions are allowed. 

When only those transitions in the directed edge set S can occur, since the 
Markov chain is irreducible, there exists a finite sequence of intermediate points 
through which one can move from i^ to ^ in m — m(r, E) < -t-oo steps: 

u = i.(0) ^ ^.(1) ^ . . . ^ ,.(™) =. e 

Consider now the piecewise linear path that moves from z^ to ^ through the 
above sequence of points with velocities such that each segment is covered in 
time T/m. Then S[o,t]{iJ-W) < Ci(T) = mCaiT/m), and the proof of the first 
bullet is complete. 

St{£,\v) < CiiT) follows immediately from (2.18), whence the second bullet 
follows. 

To sec the third bullet, we use (4.11). Since M.\(Z,) is a subset of a finite 
dimensional space, the topology of weak convergence is the same as the topology 
induced by the total variation metric. In particular, if po(^j C) ~^ then v(i) — >■ 
if(i) for every i S 2^. As a consequence, for every e > and with T = e, we can 
choose a (5 > such that each of the first four terms in (4.11) is upper bounded 
by e, and so po{v,^) < S implies 

Sei^W) <4e + Cr^e<C2e 

for some C2 < -I-cxd, and the proof of the third bullet and the Lemma is complete. 

D 

5. Proofs of Lemma 2.3 and Lemma 2.4 

We begin with two useful lemmas. 
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Lemma 5.1. Let L{t) be a matrix of rates such that the solution fi : [0,T] — !> 
Aii{Z) to the ordinary differential equation /i(i) = L{t)*fj,{t) with /i(0) = v has 
5'[o.T](/^|f^) < +00. There exists a constant K < +cx) such that 

Proof. It is easy to verify that t*(u — 1) = ulogu — u+1>m — e+1 for all 
M > 0. By Theorem 2.2, S\r)T]{lA^) < +00 implies that its evaluation is given 
by (2.14). Using these two facts, we get 






di 



di 



> 



JfO.Tl L,.^^ J 



'["'^] -(^..)6£ 



and the lemma follows. D 

The next lemma bounds the increase in the cost due to time scaling on a 
fixed path between two points. 

Lemma 5.2. Let L{t) be a matrix of rates such that the solution fi : [0,T] — >■ 
A4i{Z) to the ordinary differential equation /i(i) = L{t)* fi{t) with fi{0) — v has 
'S'[o.T](/^|^) < +00 and fJ.{T) ~ ^. Let < a < 4-cxd be a time scaling. With 
T' = T/a, consider the path {flit) ~ fi(at) \ t G [0,r']} having /i(0) — v and 
fi[T') = i. Then 

m = L{trm. ie[o,r'] 

where L(t) = aL{at). Furthermore, the scaled path fi : [0,r'] — > Aii{Z) satisfies 

VT'](Ak) < VT](/^k) + |ioga| / \ Y. {^i{t){i))uM di(b.i) 



'["■^1 {^.ms 



"i^C.T. 



a 



Proof Clearly fi{0) = fi{0) = 1/ and /i(T') = ^(aT') = ^j(r) = ^. Since /i(i) = 
L{t)*iJ.{t), we also have 

fi{t) = J^^ = afi{at) = aL{at)*n{at) = aL{at)*fi{t) 

from which L{t) ~ aL{at) is obvious. Its (i,j)th entry h,j{t) equals alij{at) 
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The cost of fl : [0,r'] -^ Mi{Z) is then 

alij {at) 



37 






1 



(»j)e£ 



dt 



\ 
[0,T' 



/ Y] (/^("i)(J)) XijiAo^t)) t\ f f .\\ 



dt 






.KjifJ-iat)) 



Aij(^(ai)) 



(log a) + 



1 — a ' 



a 



dt 



where we have used the fact that 



T*{au — 1) = a < r*(u — 1) + u(loga) + 



1 -a 



u>0. 



Changing variables from at to t and continuing, we get 



VT'](Ak) 






(ij)6£ 



\i,jiAt)) 



dt 



+ (loga) f \ E (a^W(O) ^..jW 



dt 



('j)6e 



1-a 






(»j)e£ 



Since the first term on the right-hand side above is S[o^T]il-A'^) ^-^^ ^i,ji') — C*! 
(5.1) follows. ' ' D 

Proof of Lemma 2.3. Fix T > 0. Fix an arbitrary e such that < e < T/4. 
Let (5 > be as given by part 3 of Lemma 2.2 so that /3o(i', C) < ^ implies 
SeiCW) < C2e. 

Let {{i/i,^i),i = 1,2} be two points in Z x Z. By an abuse of notation, let 
Pt given by 

Pt{{i^i,^i),{'^2,^2)) ==max{po{iyi,i'2),po{^i,£,2)}. 

denote the metric on Z x Z. Let Pt{{i^i,£,i), {1^2, £.2)) < S. We need to show that 
St{£i\j^i) and 5't(C2|j^2) are close to each other. 

Obviously, po (1^1, 1^2) < ^ and po(^2,Ci) < S- Let p denote the minimum cost 
path from 1^2 to ^2 in time T with cost S't(C2|^2)- Consider the path from vi to 
^1 as follows: 

• Traverse the path from vi to 1^2 in time [0, e], as given by part 3 of Lemma 
2.2. This traversal costs at most C2S. 
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• Given the optimal [0,T]-path fi from 1^2 to ^2, consider the sped-up path 
/i : [0,r - 2e] -^ Mi{Z) given by jl{t) = ^x{at) with a = T/{T - 2e). 
Travel from V2 to ^2 in the duration [e,T — e] along the path jl. 

• Traverse the path from ^2 to ^1 in time [0,£], again as given by part 3 of 
Lemma 2.2. This traversal's cost is also at most C2£- 

The minimum cost for traversal from vi to ^1 is at most the sum of these paths. 
Hence, by Lemmas 5.2 and 5.1, we get 

SriiiWi) < 2^26 + 5T(6k2) + (log ^,^)(5T(6l^2)+i^T) + ^Crr 

< 5t(6I^2) + 2^26 + ( j7^ - 1) {C\{T) + KT) + 2Cre 
where we used log u < u — 1 for u > 0. Observing that 

s<T/4^^--l = ^^<l£, 
' r - 2e T - 2e - T ' 

wc deduce that 

where we may take Ca{T) = 2C2 + ^K + ACi{T)/T + 2Cr. Reversing the roles 
of (i^i,Ci) and (1^2,^2)7 we deduce 

\St{^iWi) - St{^2W2)\ < C4{T)e. 

This concludes the proof that (i^, ^) t— >■ 5*71(^1;^) is uniformly continuous. D 

We now provide the proof of the result on uniform continuity of the analogous 
quantity V{£_\v). 

Proof of Lemma 2.4- Fix £ > and choose 6 as in the third part of Lemma 2.2. 
Let (j^i, ^1) and (1/2, ^2) be such that the starting points are ^-close to each other 
and so are the ending points, i.e., po{i^i,i^2) < 6 and po{^i,^2) < S. Consider 
the following path: 

• Traverse the path from vi to 1^2 in time [0, e], as given by part 3 of Lemma 
2.2. This traversal costs at most C2£- 

• Traverse by a path from 1^2 to ^2 in finite time by a path with cost at most 

V{^2W2) + C2S. 

• Traverse the path from ^2 to ^1 in time [0,e], again as given by part 3 of 
Lemma 2.2. This traversal's cost is also at most C2£- 

We then have 

^(ai^^i) < C2£ + (1^(6^2) + C2e) + C2£ - 1/(61^^2) + 3C2£. 

Reversing the roles of {vi, ^1) and (:/2, C2) and via a similar argument, we deduce 
that 

\ViClWl) - V {^2^2)1 <3C2£ 

which shows that (i^, ^) >— s- V{^\iy) is uniformly continuous. D 
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6. Proof of Theorem 2.4 

Again, we proceed through a sequence of lemmas. Let vn -^ v weakly. By The- 
orem 2.2, the sequence of laws of the terminal measure (p|^ ^^^ > 1) satisfies 
the large deviation principle with speed N and good rate function Sri^li^)- By 
Varadhan's lemma, for every / e Cb{Aii{Z)), we have 

hm llog/ e''Upl'l]^= sup IfiO-STi^W)]. (6.1) 

N^ + oo 1\ Jmi(Z) ieMiiZ) 

Let us define 

A(/|^)= sup [fiO-SriiM. (6.2) 

ieMiiZ) 

Observe that the rate function admits the characterization (see, e.g., [13, Th. 

4.4.2]) 

St{£.\v)^ sup [f{£,)-K{f\v)]. (6.3) 

feCi{Mi(Z)) 

Lemma 6.1. Let f G CbiMi{Z)). The mapping v e M\{Z) ^ A(/|i') e'R is 
continuous. 

Proof. Since / is continuous, by Lemma 2.3, the mapping 

V : {iy,0 e Mi{Z) X Mi{Z) ^ f{0 - St{C\,^) G M 

is jointly continuous. As A4i{Z) is compact, the supremum in the definition of 
(6.2) is attained. 

Let vn — > V weakly, and for each vn, let S,n denote a point where the supre- 
mum in the definition of (6.2) is attained. In other words, A{f\h'N) = vi'^Nj^N) 
for each N. The sequence {{i'n,^n),N > 1) has a convergent subsequence that 
converges to (1^,^)5 for some ^. Reindex so that we may take {vn,£,n) -^ {^'tO 
as A^ — > -l-oo. By the continuity of rj, 

HJWn) = v{'^n,^n) -> vii^iO 

as N ^ +CXO. The proof will be complete if we can show that A(/|i^) — 7]{i', ^), 
i.e., the supremum in 7](i^, •) is attained at ^. 

To see this, observe that for any ^', we have Tj{v{q,£/) < r]{iyN,^N)-, and so 

Tl{v,S,')= „lim l{vN,0 < limsup?7(i/jv,^Af) = ?7('^,C)- 

This completes the proof of the lemma. D 

Our next result show that the convergence in (6.1) is uniform. This is where 
our uniform large deviation result for nonchaotic initial conditions comes in 
handy. 

Let M\ (Z) C A^i(Z) be the subset of values taken by the empirical mea- 
sure Mjv when there are N particles. 
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Lemma 6.2. The convergence in (6.1) is uniform in the following sense: for 
each f E Cb{A4i{Z)), we have 



lim sup 



^^ JMi{Z) 



(6.4) 



Proof. We will prove this by contradiction. Suppose the above limit is not zero. 
Then there is an e > and an infinite subset Vi C N such that 



sup 



— log 



Mi(Z) 



e^f dp^J"^ - A(/|.) 



> e, for every N eYi, 



i.e., the violations occur infinitely often. So we can find a sequence {i'n)ni£Yi 
such that 



llog/ e^fdpi^J^^^MfW^: 



> e, for every iV £ Vi. 



Extract a further subsequence, which is another infinite subset V2 C Vi, such 
that {h'N)NeV2 ^ V for some v. By Lemma 6.1, (A(/|i'jv))jvgV2 ^ ^{!\^)i S'Hd 
so 



— log 



=^/ W„(^) 



dp. 



Mi{Z) 



VN,T 



A(/|^) 



> -, for all sufficiently large N £¥2- 



(6.5) 

Construct a new initial state sequence {vn)n>i that matches with the above 
subsequence for A^ G V2 and such that vn -^ ^- For such a sequence, by Theorem 
2.2 and Varadhan's lemma, (6.1) holds. But (6.5) for all sufficiently large N €¥2 
is a contradiction to (6.1). D 

Proof of Theorem 2.4- Consider the joint measure pQ j. given by 

We shall apply Feng and Kurtz's [18, Prop. 3.25]. To do this, we need to verify 
three conditions listed below. 

• Exponential tightness. The sequence {pQ-f,N > 1), which comprises of 
probability measures on the compact product space, is trivially exponen- 
tially tight. 

• Uniform, convergence of the Laplace- Varadhan functional in the initial con- 
dition. For each A^, the probability measure pi, j. is supported on the com- 
pact subset M.\ (Z). By Lemma 6.2, for each / S Cb{M.i{Z)), the con- 
vergence of the Laplace- Varadhan functional is uniform, as given in (6.4). 
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• Continuity of Laplace- Varadhan functional in the initial condition. For 
each / G Cb{Mi{Z)), the function u ^^ A(/|i^) is continuous, by Lemma 
6.1. 

Under the above conditions, Feng and Kurtz demonstrated in [18, Prop. 3.25 
and Rem. 3.26] that if the first marginal sequence (pQ , A^ > 1) satisfies the 
large deviation principle with speed N and good rate function s, then so does 
the sequence of joint laws (pg j,, iV > 1) with good rate function So.t{i^,S,) = 
s{iy) + St{S,W)- This concludes the proof. D 

Appendix A: Multiple Equilibria 

In this appendix, we verify that the classical program of Freidlin-Wentzell [21, 
Ch. 6] can be extended to our setting. There are primarily two things to keep in 
mind. First, for all finite N, we have a jump process on the simplex. Second, the 
quantity 1^(-|-) defined in (2.19) is only uniformly continuous and not Lipschitz 
continuous. But this uniform continuity suffices. Though the changes are minor, 
wc provide the entire sequence of lemmas with modified proofs for completeness 
and ease of verification. 

A.l. Auxiliary results 

We begin with a subset of the auxiliary results in [21, Ch. 6] that were shown 
for diffusions on a compact manifold. 

Lemma A.l. (Freidlin and Wentzell [21, Ch.6, Lemma 1.2]). For any e > and 
any compact set K C Mi{Z), there exists a Tq such that for any i/, ^ G X there 
exists a function n{t),t G [0,T],/z(0) = v,ii{T) = ^,T < Tq with S\qt]{iA^) < 

Proof. Fix e > 0. By part 3 of Lemma 2.2, with the constant C2 as in that 
Lemma, and £1 = £/(4C2), there is a (Si G (0,ei) such that two points within 
a distance 5i can be connected by a path of duration £1 and cost at most 
Czei = e/4. 

By Lemma 2.4, there is a 82 such that 

Ta&y.{pf){£,i,^2),Po{vi,V2)} < h implies \V{^2W2) ~ V"(^i|i/i)| < e/4. 

Let 6 = min{(5i, 62}. Choose a finite 6-net {vi} of points in K . Connect them 
with curves fj,ij(t),t G [0,Tij], fiij(0) = Ui, fiij(Tij) = Vj such that 

S[o.,n,]{^i^Av^) <V{vj\v,)+e/4.. 

For arbitrary i', ^ G A', let Vk and vi be the respective closest points on the net. 
We can now find a path from i^ to z^fe, then to vi along p.k,i, and then to ^, with 
overall cost at most 

e/4 + {V{vi\uk) + e/4) + e/4 = V{vi\vk) + 3e/4 < V{i\v) + e 
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where the last inequahty foUows from the uniform continuity in Lemma 2.4 and 
the choice of S. The duration of the path is Tkj + 2ei < Tq := (maxijTij) + 
2si. ' ' D 

For a set A C Mi{Z), let [A]s denote the (open) (5- neighborhood of A. 
Its closure will be denoted [A]s. For the following lemma, recall the notion of 
equivalence between two points: i/ ^ ^ if V{£,\i') = V{i'\(,) = (see Section 2.7). 

Lemma A. 2. (Freidlin and Wentzell [21, Ch.6, Lemma 1.6]) Let all points 
of a compact set K he equivalent to each other, hut not to any other point in 
Mi{Z). For any e > 0, (5 > 0, i^, ^ G K, there exists a T > and a function 
/x(t),0 < t < T with /i(0) = ly^niT) = ^,^t(t) e [K]s for all t e [0,T], and 
S[o,T]itJ-W) < £• 

Proof. Fix e > 0,6 > 0,1^,^ Q K. We can find a sequence (r„, n > 1) and paths 
/i(") : [0,T„] -^ Mi{Z) such that /x(")(0) = v,lJ.^''\Tn) = £, for aU n > 1, and 
e > S'[o,t„](m^"^I'^) — > as n — > +oo. Observe that Mi{Z) \ [K]s is compact, 
and so if an infinite number of it-""' left [K]^, there is a limit point z outside 
[K]s. Using part 3 of Lemma 2.2, and ^[o^TnlCM^"^!^) — i> as n — > +oo, it 
follows that V{z\v) = V{^\z) ~ 0. Together with X^(i^|^) — 0, we conclude that 
V{i'\z) = and so z -^ i^. But then z is an equivalent point outside K , which is 
a contradiction. 

Hence /i^"^ goes outside [K]s for finitely many n. Take the first index larger 
than these. The corresponding path remains completely within [K]s, and meets 
all the other requirements. D 

In this section, we shall use the notation 

TA:=inf{t>0 I ^lNit)<^A}. 

The law for this exit time depends on N and ly through the law pj> for fi^. This 
dependence will be assumed as understood and will be suppressed for brevity. 

Lemma A. 3. (Freidlin and WentzeU [21, Ch.6, Lemma 1.7]). Let all points of 
a compact set K he equivalent to each other and let K ^ A4i(Z). For a S > 0, 
let 

T[KU ■■= inf{t > I ^iNit) i [A']4. 

For any e > 0, there exists a 6 > such that for all sufficiently large N and all 
V G [I^]s, we have 

where the expectation is with respect to the measure pi, . 



E[rr^i J < e+^^ 



Proof. Fix £ > 0. Again by part 3 of Lemma 2.2, there is a 5i > such that two 
points <5i-close have a path connecting them of duration ei = e/(4C2) and cost at 
most e/4. Choose S, outside K such that po{^,K) < Si. Choose S < po{^,K)/2; 
we thus have < S < po{^, K)/2 < po{^, K) < Si. 

Consider a finite i5-net of K. Lemma A.l assures existence of paths that 
connect any pair of the net with cost at most e/4. Let Tg denote the maximum 
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time among these paths, where the m.axim.um is over pairs belonging to the net, 
and let Tq = TQ + 2ei. Traverse from any i' S [K]s to its nearest point on the net, 
then traverse from that point to the point on the net nearest to ^, and thence 
to ^. Now extend this path following the McKean-Vlasov dynamics so that the 
total duration is now Tq. This last appendage incurs no additional cost. Denote 
by jj, the resulting path of duration Tq. Clearly S'[o,To](a*I^) ^ 3e/4. 

Now, any trajectory that is strictly 5-close to the trajectory /i exits [K]s at 
least once in the interval [0,To] because for some t £ [0,To], we have ^(t) = ^ 
which is at a distance greater than 2S from K. We then have 



> e~^^^/'', for ah zy e [K]s, for all N > some Nq, 
where the last inequality holds by (2.17) in Corollary 2.1. Consequently 

pi'''' {t[k]s >To}<1- e-3^^/4, for ah ly 6 [A^,, N > Nq. 
This uniform bound, the Markov property, and induction imply 



from which we obtain 

E[r[K] J < To 5] (1 - e-3e^/4)™ ^ y^^3e^/4 < ^sN 
m>0 

where the last inequality holds for all sufficiently large N. This concludes the 
proof. n 

Lemma A. 4. (Freidlin and Wentzell [21, Ch.6, Lemma 1.8]). Let K he an 
arbitrary compact subset of A4i{Z) and let G be a neighborhood of K . For any 
e > 0, there exists a S > such that for all sufficiently large N and all v 
belonging to g, with g = [K]s and g = [K]s, we have 



Ig (/ijv(O) dt 

[O.tg] 



> e-^, 



where the expectation is with respect to the measure pi, . 

Proof. Fix e > 0. Choose 5i small enough so that [K]s^ C G. Next, choose 
§2 as in part 3 of Lemma 2.2 so that with ei = e/{2C2), any two (52-close 
points can be connected by a path of duration ei and cost at most e/2. Now let 
5 < min{(5i, (52/2} and set g = [K]s. 

Fix T > El. Take any v £'g. Connect it to the closest point on K (via a 
path of duration ei and cost < e/2) and then extend via the McKean-Vlasov 
path with this initial condition for a further duration of T — ei. Call the entire 
path of duration T as /i. So long as ^ is inside the (5/3 neighborhood of K, any 
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(5/3 neighborhood of /i hes completely inside g — [K]s. By assumption (A), the 
McKean-Vlasov dynamics has a bounded velocity field. Consequently, the part 
of fi that begins at K and until either its exit from [iiTJ^/s or time T, whichever 
occurs earlier, is of duration at least to for some to > 0, independent of the 
starting point. It follows that 

{prip-N^iA < S/3} implies {r^ > minjT, toll- 
Furthermore, TG > Tg and lg(/xjv(i)) = 1 until the random path exits g. Thus 



lff(MAf(0) dt 

[O.tg] 



> 


E[rg] 


> 


E[r<,.l{pT(Mw,M)<V3|] 


> 


min{r,to| -pi,^) {PT[m,y) < s/3} 


> 


min{r,to| -6-^^/2 (by (2.17)) 


> 


e-^^ 



where the last two inequalities hold for all sufficiently large N. D 

Lemma A. 5. (Freidlin and Wentzell [21, Ch.6, Lemma 1.9]). Let K he a com- 
pact subset of M.i{Z) not containing any w -limit set entirely. There exist positive 
constants c and To such that for all sufficiently large N , any T > To, and any 
u € K, we have 



p\f'>{TK >T}< 



e 



Proof. For a sufficiently small S, the closed (5-neighborhood of if, denoted [K]s, 
does not contain any w-limit set entirely. Indeed, if this were not true, we can 
find a sequence of (5 J, such that each set in the nested decreasing sequence of 
sets [K]s contains an w-limit set entirely. For a S, define n{S) to be the closure of 
the union of w-limit sets contained in [K]s. Clearly, n{S) is a positively invariant 
set, and the family indexed by 5 is a nested decreasing sequence of nonempty 
compact sets. Then ns^{S) is a nonempty compact invariant set in K. Further, 
it contains an w-limit set entirely, a contradiction. 

For f € [K]s, denote by t{i') the time for first exit of the solution to the 
McKean-Vlasov equation with initial condition v from the set [K]s. Since [K]s 
does not contain any w-limit set entirely, t{v) < +00 for all u S [K]s. The 
function T{h') is upper scmicontinuous, and consequently, it attains its largest 
value max^ ■rj^T(i^) = Ti < +00. 

Set To = Ti + 1 and consider all paths of duration To that take values only in 



[K]s. It is easy to see that this set is closed. It follows that for each ly G [K]s, we 
have that S[o,To]{'W) attains its minimum A^v) on this set. Further, the mapping 
u I— >■ A(^/) is continuous, as can be shown by an easy application of Lemmas 2.2, 
5.1 and 5.2. Thus A :— min jttt- A{iy) is attained. This minimum is strictly 
positive since there are no trajectories of the McKean-Vlasov equation among 
the paths under consideration. 
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Fix e < A, a. v ^ K, and consider the family of paths 

$,(A - e/2) = {^ : [0, To] ^ Mi{Z) \ 5[o,To](mI^) <A- e/2}. 



Any path in this set exits [K]s in the interval [0,To]. With initial state v, the 
event {tk > To} implies that the trajectory remains entirely within K, and 
since any path in ^^(A — e/2) exits [K]s, wc must have 

PTo(AiJV,$^(A-e/2)) >S. 

It follows that 

pW{r^>To} < pW{pT„(/^iv,*.(A-£/2))>J}. 

By considering any vi £ K, and by using (2.16) of Corollary 2.1, we have 

sup pi'^^lTK > To} < sup p^f {pTo(/^iv, ^M - e/2)) > 6} 

< exp{-iV(inf{5[o,T](M|i^i)lM G $.,(A~£/2)}-e/2)} 

for all sufRciently large N 

< e~^' •* for all sufficiently large A^. 

For our fixed ly G K, the Markov property then implies 
Pi""^ {rx > (m + l)ro} < m{TK > mTo} ■ E[1{tk > To} \ //^(mro)]] 

< pL"^ {tk > mTo} ■ ( sup p(,f ) {tk > To}' 

< pI^Htk > mTo} ■ e-^^^-'\ 
By induction, for a T > To, wc have 



.W{r,,>T} < pi^UTK> 



To 



T 

for c = (A — s)/To, and this completes the proof. D 

The above theorem has the following immediate corollary. 

Corollary A.l. (Frcidlin and WentzcU [21, Ch.6, Corollary to Lemma 1.9]). 
Let K be a compact set not containing any uj-limit set entirely. There exists a 
positive integer No and a positive constant c such that for N > No and any 
u G K, we have 

E[tk] < To + l/(cA^o) 

where the expectation is with respect to the measure pi, . 
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Recall the definition of V given in (2.19), and the notion of equivalence on 
Mi{2) given in Section 2.7. Under condition (B) in Section 2.7, we have equiv- 
alent sets Ki , . . . ,Ki to which all w-limit sets converge. We shall now define a 
discrete-time Markov chain of states at hitting times of neighborhoods of these 
compact sets. In order to bound the transition probabilities of this chain, recall 
the definitions of V{Ki,Kj) given in (2.34) and V{Ki,Kj) given in (2.35). 

Define the following quantities: 



• 



An To such that < ro < (1/2) minij po{Ki, Kj), 



• An ri such that < ri < tq. 



The set C as C := MiiZ) \ Uj^Jii:,]„ 
The set r, as T, := [K,]r,, 
The set gt as gt := [Kijr-^, and finally. 
The set g as g := u'^^^i. 



Let us now define the following stopping times: 

• To := 0, 

• The time for exit from the union of the rp neighborhoods of the compact 
sets Ki's, i.e., cr„ :— ini{t > t„ | fi^it) € C}, 

• The time to re-enter g, i.e., r„ :— inf{t > (T„ | fiN{t) G g}. 

Finally, we define Zn '■— lJ,N{Tn)- We shall use the notation p^^\v,'gj) for 
pi (MAr(T„) g J]) when /XAr(T„_i) = V. 

Lemma A. 6. (Freidlin and Wentzell [21, Ch.6, Lemma 2.1]). For any e > 0, 
there is a small enough tq > such that for any r2 satisfying < r2 < rg, 
there is an ri satisfying < ri < r2 such that for all sufficiently large N , for 
all V G [iCi]r2j ihe one-step transition probabilities of Z^ satisfy 

eM-NiViK,,K,) +e)} </^\i^,gj) < exp{-N{V{K,,Kj) - e)}. 

Proof. For pairs with V{Ki, Kj) = +oo, there is no smooth curve from Ki to Kj 
without touching one of the other compact sets. It follows that for any arbitrary 
< ri < r2 < ro, for all sufficiently large N, there is no path in A^![ (Z) from 
[A'ijra to gj without touching [ATi'Jro for some i' ^ i,j. Thus, for all sufficiently 
large A^, we have p^^Hi^,gj) = 0. The validity of the lemma is obvious for such 
pairs. 

For all other pairs V{Ki,Kj) < Vq for some Vq < +oo. 

Let us first argue the lower bound. Fix e > 0. Choose S as in part 3 of Lemma 
2.2 so that with ei = e/(10C2), any two points J-close can be connected by a 
path of duration ei and cost at most e/10. Set 

ro = min{(5/2, (1/3) minpo{Ki, Kj)}. 

Fix arbitrary r2 satisfying < r2 < ro. 

For each (i, j) with V{Ki,Kj) < -l-oo, choose paths /x''^ : [0,Tij] -> Mi{Z) 
such that 
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n 



/i*'^(f) does not touch Ui'^ijKi' for t e [0,7!^^], and 
. 5[o,t](m*'^Im''^(0)) < V{K,,K,) + 0.2e. 

Now choose ri so that 

< mm{r2, ^, -min{po (^"'(0, U.'^.,^ AV) | t e [0,T,,,)], 1 < i,j < z}}. 

Also choose S' < min{ro — 7'2,fi} so that r2 + 6' < tq and hence tq + S' < 2ro < 5 
by the choice of S. 

Take any ly e [ATiJ^a- Fix a finite J'-nct of Ki. If i ^ j, consider the following 
path. 

• Connect ly to the nearest point vi e Ki with a path of duration ei and 
cost at most O.le. 

• Connect i^i to the nearest point 1^2 on the (5'-net of i^T^ again with a path 
of duration ei and cost at most O.le. 

• Let 1^3 be the point on the 6' -net nearest to /i*'^ (0). Traverse the path given 
by Lemma A. 2 that connects 1/2 to 1^3 without leaving the ri-neighborhood 
oi Ki. Thanks to the finite number of points on the S'-net, this can be done 
in bounded time. Moreover, the cost is at most O.le. 

• Connect 1/3 to /i'^-'(O) with path of duration £1 and cost at most O.le. 

• Then traverse the path given by /i*'^ . 

Hi = J, then simply take i^ to a point at a distance rg + S' from Ki and then to 
the nearest point in K^. Note that tq + S' < S, and so the duration of this path 
is 2£i and cost at most 0.2e. The constructed path is of bounded time duration, 
bounded say by Tq. We can thus extend all paths to duration Tq along the 
McKean-Vlasov path, an appendage that incurs no additional cost. Call the 
resulting path fJ.{t),t e [0,To]. Clearly, 

S[o,ToMiy)<ViK,,Kj) + 0.6e. 

If pToi^J■N 1 fJ-) < S' 1 then the trajectory fij^ begins at a point v within an rg- 
neighborhood of Ki, reaches the ^'-neighborhood of Kj and so hits g ■ = [ATj-Jr-j , 
is at most S' distance away from the trajectory fi, and hence does not hit the 
rg-neighborhood of any Ki>,i' ^ i,j; then fiN{Tn) G 5,- In other words. 



and so 



{PTa{f^N,f^) < S'} C {l-lN{Tn) G 9j}, 



p'^'^H'^.g,) > /J'HpToim.f^XS'} 



> 



exp{-iV(5[g,To](/i|^) + 0.1e)} 



> exp{-N{V{K,,K,)+e)}, 

where the second inequality holds for all sufficiently large N uniformly over the 
initial condition, thanks to (2.17) of Corollary 2.1. This establishes the lower 
bound. 
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Wc now prove the upper bound. Consider any path fi of some duration T 
starting at v in the ri-neighborhood of Ki, ending at a point say ^ in the 6'- 
neighborhood of g^ at time T, and not touching any of the other compact sets 
Kii,i' ^ i,j. By the choices of ri and 6', there are short paths from a point 
p' € Ki to f and from ^ to a point ^' £ Kj, each of duration ei and cost at most 
O.le. The path that traverses from v' to v, and then along fj. to ^, and thence 
to ^', has cost at most S'[o.t](mI^) + 0-2^ > V{Ki, Kj), and so 

S[o,TMiy)>V{K,,K^)~0.2e. (A.l) 

The same holds for any path fj, of some duration T starting at v in the ri- 
neighborhood of Ki, touching the ^'-neighborhood of g- at time in [0,T], but 
not touching any of the other compact sets Ki', i' ^ i,j. 

By Lemma A. 5, with the set C in place of K, a set that does not contain any 
w-limit set entirely, and with Ti = Tq + Vq/c where c, Tq are as specified in that 
lemma, we obtain 

pW{ri > Ti} < sup pi':'^{Tc >Ti}< e-^^" (A.2) 

for all sufficiently large N. 



Consider a trajectory fj.N with /^Ar(0) = i^ G [Kiln and /iAr(Ti) £ g^. There 
are two possibilities: (1) ri > Ti, or (2) ti < Ti in which case the trajectory 
enters g- in [0,ri]. In this second case, with 

*[o,Ti],.(f) := {a* : [0,ri] ^ Mi{Z) \ 5[o,T](/^|f^) < «}, 

we have 

PTA^^N,^[o,n]AyiK^,KJ) - 0.3£)) > S'. (A.3) 

To see this, note the conditions 6' < ri, ti < Ti, and fJ.N{Ti) G g,,. If /i is any 
trajectory satisfying pr^^diN, fJ-) < S' , then fi must hit the ^'-neighborhood of 
g- without touching any of the other compact sets Ki',i' ^ i,j. From (A.l), 
subtracting an extra O.le, we get S[Q_Ti]{fJ-W) > V{Ki, Kj) — 0.3e. By contrapo- 
sition, under the noted conditions, any /i with S'[o.Ti](/^|'^) ^ ^(^ii^j) ^ 0-3£ 
must satisfy pTi{p-N,P-) > <5', and hence (A.3) follows. 
Putting the two cases together, we get 

pi''Hm{ri)€g,} < pW{ri>Ti} 

+ P^^HpTAf^N,^[o.T,]jViK,,K,) - 0.3£)) > ,5'} 

(a) 

< e-^^° + exp{~N{V{K^,Kj) - O.Se) + iV(O.le)} 

< cxp{~N{V{K„K,)-e)}. 

In the above sequence of inequalities, (a) holds for all sufficiently large N due 
to (A.2), (2.16) of Corollary 2.1, and the definition of $[o,Ti],iv Inequality (b) 
also holds for all sufficiently large N because V{Ki,Kj) < Vq. This proves the 
upper bound and completes the proof. D 



Borkar and Sundaresan/Invariant Measure in Mean Field Models 49 

Recall the definition G{i} in the paragraph preceding (2.36), the definition 
of W{Ki) in (2.36), and the definition oi Si,i = 1, . . . , /. We are now ready to 
state the main theorem of this appendix. 

Theorem A.l. (Freidhn and Wentzell [21, Ch.6, Theorem 4.1]). Assume (A) 
and (B) hold. For any e > 0, there is an arbitrarily small ri > such that 

exp{-iV(s,+e)} < p^^Ht^'drJ <cxp{-iV(s,-e)} 

where Si are defined in (2.37). 

Proof. All the steps of the proof of [21, Ch.6, Th.4.1] hold, since the analogs of 
all the lemmas used in that proof have now been verified to hold. D 
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